← Back to blog

The Death of Documentation

The Death of Documentation
TL;DR

Static docs rot into folklore, and agents read them as ground truth. Generate docs just in time from code, and hand-write only ADRs: the why code cannot hold.

Earlier this year I gave a talk at AI Engineer Melbourne called The Death of Documentation. This post is the same argument in written form. There is some irony in turning a talk about the end of static documents into a static document, but the argument explains itself: some things still deserve to be written down by a person, and by the end I hope you will agree on which things those are.

We inherited the blueprint

Software documentation is a habit we inherited from traditional engineering. When you are pouring concrete, mistakes are expensive, so the blueprint comes first and the building follows it. Early computing had the same economics. Machine time was booked in advance, a bad punch card wasted a day, and specifying everything up front was cheaper than discovering problems at run time. Writing the document before and alongside the system was a rational response to the cost of being wrong.

That cost structure died decades ago. The habit outlived it.

Documentation decays into folklore

I have been building software for fifteen years, and I have watched the same lifecycle play out on every team I have worked with. Documentation is written in a burst of good intentions, usually near a launch. Then deadlines arrive, and documentation is the first casualty. The code keeps moving. The document does not. Within a few quarters the wiki is not a description of the system; it is folklore, a rumor about how the system used to work.

Humans cope with this surprisingly well. We read the stale page, notice it contradicts the code, shrug, and go ask whoever has been on the team longest. The stale document costs us a little time and some trust, and the social fabric of the team absorbs the rest.

Agents make stale docs expensive

AI agents do not have that social fabric. An agent handed a codebase and a documentation folder treats both as ground truth. When they disagree, it cannot lean over a desk and ask which one to believe. It either burns tokens trying to reconcile the contradiction or, worse, it acts on the document. A human misled by stale docs loses an afternoon. An agent misled by stale docs ships the mistake.

So the arrival of agents changes the stakes of a problem we had learned to live with. Stale documentation used to be a tax. In a workflow where agents read your repo and act on it, it is a liability.

Prose is lossy compression

Here is the part that took me longest to accept. The problem is not that we write documentation badly. The problem is the medium. Code, commit history, API schemas, and configuration are structured, deterministic data. Describing them in English is a lossy compression algorithm. Detail falls away no matter how careful the author is, the way detail falls away when you sketch a CAD model freehand. You cannot rebuild the model from the sketch.

For most of computing history that loss was the price of admission, because English was the only format the audience could read. That assumption just quietly stopped being true.

The model can read the code

In the talk I ran a small experiment: I gave a model a chunk of minified JavaScript, single-letter variables and all, and asked what it did. It correctly worked out that one variable held a user’s account balance, walked through the transaction logic, and identified the payment gateway integration, with no comments and no prose in sight. The explanatory paragraph I would once have written above that function adds nothing the model cannot recover from the source.

If the reader can consume the deterministic artifact directly, the lossy translation of it is not just redundant. It is a second source of truth waiting to drift from the first.

We automated the wrong problem first

At Prefactor we did what most teams are doing right now: we automated the writing. We point an LLM at Linear tickets and GitHub pull requests, generate markdown, and a human reviews it before it lands. It works, and it is far cheaper than writing by hand. It is also, I now think, solving the wrong problem. Generated prose is still prose. It still goes stale the moment the next PR merges, and it still compresses structured data into a lossy format. We made the wrong artifact cheaper to produce.

Two hours to integrate

The strongest evidence I have is an anecdote from the conference itself. An attendee challenged an agent to build an integration with Prefactor using nothing but our open-source SDK code and the specifications that ship with it. No tutorials, no guides. Two hours later the integration worked. The deterministic spec plus the source outperformed anything we could have written for that developer, because it was rendered for exactly their situation, on demand, from artifacts that cannot drift.

Documentation as a just-in-time service

That is the shape of what replaces the static doc: synthesis at read time. Same codebase, same specs, different renders for different readers. A new hire on day one asks for an overview and gets three boxes, two arrows, and who owns what. A security auditor asks the same system the same afternoon and gets a data-flow diagram with PII annotations, encryption status, and compliance boundaries. Neither document existed until the moment it was requested, and neither can be stale, because each was generated from the current state of the system.

What the model cannot recover

There is a hard limit to all of this, and it is where the talk stopped being a eulogy. A model reading your repo can recover what the system does and how it works. It cannot recover why.

My favorite example: an agent audits your stack and correctly observes that Postgres would suit your workload better than MySQL. The analysis is right. What the agent cannot know is that you are locked into a five-year vendor contract, that the CTO vetoed the migration on cost, and that a data residency requirement rules out the managed alternative. Those constraints are real, load-bearing, and completely absent from the repository. They live in meetings, contracts, and inboxes. No amount of reading the code recovers them, and an agent acting without them will confidently propose the audit trail-visible mistake you already decided not to make.

Write the decisions down

Which is why the one document I am asking you to keep writing by hand is the architecture decision record. An ADR captures the date, the options considered, the option chosen, the constraints that drove the choice, and the conditions under which it should be revisited. It is the only category of documentation a machine cannot generate, because the source data never reaches the repo. Everything else about your system is recoverable from code. Intent is not.

The new division of labor

So the split I proposed in Melbourne is this. Machines trace execution paths, generate the diagrams, write the SDKs and CLIs, and answer questions about the system on demand, rendered for whoever is asking. Humans record intent: the strategy, the constraints, the business and legal and political context that shaped each decision. Machines own the what and the how. Humans own the why.

Documentation as we knew it, the static relic of a time before AI, is gone. Documentation as a dynamic service, rendered just in time from code and context, is only getting started. If you want the full version with the demos, the talk is up on Conffab.

See how every agent performs, and make it better

Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.