Field notes

Dark Code: the new dark data problem

An editor pane of mostly-dark code with a comprehension beam sweeping across, lighting a few lines whose meaning connects to a semantic anchor.

Twenty years ago, the volume of enterprise data outpaced any organization’s ability to know what it meant. Fields existed with no owner, no definition, and no lineage. Spreadsheets accumulated in shared drives, and data warehouses grew faster than the documentation around them. We called the resulting condition “dark data,” and we spent the next decade building the response: semantic layers, catalogs, lineage tools, data contracts, and governance frameworks. The cost of dark data is well understood now, and so is the fix.

We are about to do the same thing to our codebases.

AI authorship is generating code at rates no human review process was designed to handle. The output compiles, it passes tests (including the tests it wrote for itself), and it looks fine in pull request reviews. But six months later, when something breaks at 3am, the person paged has no anchor to interpret what they are looking at. There is no author to ask, the intent is not documented anywhere, and the failure modes were never considered because no one was there to consider them.

This is dark code. The semantic gap is the same as it was for data. The artifact is different. The data industry took ten years to figure out what its version cost, and the code version is going to arrive faster because the rate of generation is higher and because we are already behind.

The data problem, briefly

It’s worth being precise about what got solved with data, because it tells us what to build for code.

The data problem was never that there was too much data. The problem was that the data arrived without the artifacts humans use to make sense of each other’s work. Fields carried no documented meaning, tables had no owner, and reports were built on assumptions no one wrote down. By the time anyone needed to trace why a number looked wrong, the context that would have explained it was already gone.

The fix wasn’t less data. The fix was a semantic layer built on top of the data. Catalogs named things and said what they meant. Lineage tools tracked where values came from. Data contracts formalized the relationship between producers and consumers. Governance frameworks made ownership explicit before data shipped, rather than after it broke.

The response to dark data was never less data — it was a semantic layer that made meaning survive the moment it was created.

The pattern was the same across every flavor of the response: make the meaning of the artifact survive the moment it was created. That’s what got built, and that’s what is missing for code right now.

What dark code actually looks like

In a team that has gone fast on AI coding tools without thinking carefully about the artifacts around them, you tend to find some combination of the following:

  • Modules that ship with no clear author, because the author was an AI agent and no human took ownership at review time.
  • Functions whose intent isn’t documented anywhere outside the prompt that generated them, and the prompt wasn’t saved.
  • Test suites that pass because the AI wrote both the code and the tests against the same set of assumptions, including the assumptions that were wrong.
  • Architectural decisions made implicitly by the model’s defaults, captured nowhere, and surfacing later as constraints no one remembers choosing.
  • Failure modes that were never considered, because considering failure modes is a human discipline and nothing in the pipeline forced it to happen.
Dark code is code shipped without the artifacts a human needs to interpret it — and the bill comes due at 3am.

None of this is hypothetical. It’s the predictable outcome of treating AI coding tools as productivity multipliers without treating their output as the new substrate it actually is. We have done this before with data, the result was called dark data, and it took a decade and billions of dollars to climb out.

What this costs

Dark data did not become an enterprise priority because someone in IT was annoyed by it. It became a priority because the bill arrived in three forms: regulatory fines that ran into the billions, operational incidents that took down revenue-critical systems, and strategic decisions made on numbers no one could trust. By the time the C-suite was paying attention, the catch-up cost was enormous.

Dark code will arrive in the same three forms, with a fourth on top.

Regulatory and audit exposure. The frameworks being drafted right now, including the EU AI Act, NIST AI RMF revisions, and sector-specific guidance from the SEC, FDA, and FFIEC, all converge on the same demand: prove who authored what, with what intent, against what tests. Organizations without those artifacts will produce them under deadline, by hand, at a multiple of what proactive construction would have cost. In regulated industries, the failure mode isn’t a fine, it’s a finding that blocks a release.

Operational risk. Code shipped without documented intent generates incidents that take longer to diagnose, longer to fix, and longer to write a post-mortem for. If your mean time to resolution is moving the wrong direction as AI authorship scales, that’s a leading indicator, not a coincidence.

Strategic risk. You cannot improve what you cannot measure. If your team can’t tell which tools, prompts, or methodologies produce code that actually holds up in production, every AI investment decision becomes a judgment call made without data. That works at small scale, but it doesn’t survive a board review.

A fourth: the inability to defend the work. When a CFO asks why AI tooling spend is up sharply year-over-year and your engineering organization can’t show what was produced, by whom, at what quality, the conversation goes somewhere uncomfortable. Most organizations are getting away with this right now because the spend is still small relative to other lines on the budget. That window is closing.

The good news, and it is genuinely good news, is that the data industry already solved its version of this problem. The catalogs, the contracts, the lineage tooling, and the governance frameworks aren’t exotic, they don’t require new science, and they all have a known return profile. The same pattern applies to code. The organizations that build the semantic layer now will spend less on it, in total, than the ones that wait to be forced.

Three lenses for thinking about code semantics

When you start looking at a codebase through the dark-code lens, three questions matter more than the others.

Structural tools you already have; semantic meaning is what AI-authored code throws away; the comprehension gate is the one teams skip first.

Structural. Where does this code live, and what does it depend on? This is the question your existing tools mostly already answer through SBOMs, dependency graphs, and code search. Necessary, but not sufficient.

Semantic. What does this code actually do, what are its failure modes, and what are its contracts with the rest of the system? This is the question most teams cannot answer for AI-authored code today. The information existed at generation time, in the prompt and the model’s reasoning, and it was thrown away.

Comprehension Gate. Can a senior engineer verify intent before it ships? This is the cultural question, and it is the one organizations get wrong first. If your review process treats AI-authored code the same way it treats any other code, you aren’t running a comprehension gate. You’re running a typo check.

These three lenses aren’t new. Engineering teams have been applying versions of them for years. What is new is that AI coding tools blow through all three by default, faster than the surrounding process can adapt.

What to build

None of this is an argument against AI coding tools. We use them across Intevity, and we’re not going back. The argument is that the artifacts around them haven’t caught up, and building those artifacts is going to be the real engineering work of the next two years.

The patterns we keep coming back to:

  • Traceability that travels with the code. Every requirement gets an ID, every commit links back to it, and every test links back to both. Reconstructing intent after the fact is nearly impossible, and capturing it at generation time is cheap.
  • Decision rationale captured, not reconstructed. When the AI proposes a design and a human accepts it, the reasoning behind the acceptance gets recorded alongside the code. This is the cheapest part of the process to skip and the most expensive to recover later.
  • Acceptance test matrices, not just passing tests. A matrix that says “this requirement is satisfied by this code, verified by this test” is a semantic layer for code. It’s also what regulators are going to ask for, sooner than most teams expect.
  • Persistent context, not session memory. A knowledge graph of architectural decisions, constraints, and resolved issues that survives across sessions and developers. New team members inherit it, the model has access to it, and no one has to start from scratch.
  • Observability for the AI itself. What tools generated what code, what that cost, and what the defect density looks like by tool. Without this, you cannot measure your way out of the problem. With it, you have a starting point.
None of these are exotic — they are the direct analogs of the catalogs, contracts, and lineage tools the data industry already built.

None of these patterns are exotic. Most of them are direct analogs of what the data industry built to climb out of its version of the same hole. The catalogs, the contracts, the lineage tools. We know what they look like. We just have to build them for code.

Why this matters now

Two things are happening at once.

AI authorship is accelerating faster than human comprehension can keep up. The widening gap between them is dark code — and regulation is arriving inside it.

The first is that AI authorship is accelerating faster than organizational comprehension can keep up. The volume of code produced by AI coding tools inside the teams that have adopted them has grown sharply this year and will grow sharply again next year. There is no version of that trajectory where the surrounding artifacts catch up on their own.

The second is that regulators are starting to notice. The frameworks that will govern AI-authored code in regulated industries are being drafted right now, and they will demand exactly the artifacts dark code lacks: traceability, intent, sign-off, and accountability. Organizations that have built the semantic layer will be ready. Organizations that haven’t will be building it under a deadline set by someone else.

Code without a semantic layer is the new dark data. Organizations already know what dark data cost them. The challenge is solving for dark code before it becomes the same kind of problem.

That work is most of what Intevity Labs is going to publish. Sentinel is one piece of it, the control layer for the AI coding tool your team is most likely running today. Trove is another, the observability layer that lets you see across every AI coding tool at once. Several more tools are in flight, all pointed at the same underlying problem.

If your team is moving fast on AI coding tools and the question of what happens when this breaks in six months hasn’t come up yet, it likely will.

Intevity
More from Field notes