A codebase I worked within had accumulated more than a hundred markdown files of research, plans, investigations and more: all sitting within a docs folder. It was diligently maintained by another developer’s AI agent with an INDEX.md that spanned hundreds of lines.
As I watched my agent work in the repo I kept noticing the same behaviour from it. During a planning session or when executing on a task, it would occasionally reference one or two files from within the docs folder, read them and treat their contents as fact. This was then used to shape my current task further or included in the reasoning of plans.
I never pointed the agent towards the folder, and it never checked with me if anything in it was accurate. I wasn’t even sure of all the content it contained myself. It found the files, believed them and carried on with the task at hand.
Once I noticed this I opened some of the files it’d read. I quickly realised that they were a combination of open questions, suggestions and verbose break-downs of issues or problems we’d experienced over the past month. Some were out of date, others were incorrect and overall one thing was clear: they were all generated artefacts written, committed and attributed to another agent.
That reasoning was seeping into planning and sparring sessions or being used to answer me when I wanted to know about another part of the mono-repository I wasn’t familiar with.
I started correcting it, and every time it happened it cost another round trip. "Ignore the docs folder and focus on the current code" or "the state today is ..., disregard the docs folder". Attention and time spent several times a day, distracting me from my task, forcing me to correct the plans by hand and ultimately seeding small amounts of doubt into the agent’s ability.
Eventually, I determined it might just be best to delete the folder from my local disk. Cut off its ability to read it. Immediately the behaviour I’d observed over the previous sessions that month stopped. I’d just removed well over a hundred documents from my agent’s reach and session by session it became clear that I’d lost nothing when it came to its performance or ability.
The documents were cheap, so nobody counted them
The reasoning for building that docs folder was explained as such: producing a research artefact for this issue, decision or behaviour we have today is basically free. It’ll help us create a timeline and a knowledge base, living next to our code. Everyone else’s agents will use it to become more informed and as they contribute to and maintain it, the better everyone’s next session will be so therefore our product will improve.
So in principle the files existed for a valid reason. We’ve argued ourselves that context beats prompting and I still think that holds. I’ve found that agents able to consume your architecture and conventions reason better and make far better inference and suggestions as opposed to when they are guessing from files they happen across during their turn.
Reasoning for the docs folder sounded sensible and it was certainly well intentioned, but it relied on an unspoken assumption that the documents would stay accurate and up to date. Nobody was actually checking whether they were. These were artefacts generated by asking an agent to research a problem or topic and then committed verbatim. The committer never edited them and reviewers appeared to be skimming over the files: there was no human deciding whether the contents held value.
We’ve been here before too with lines of code and commits. Very few would defend either as a measure of productivity anymore, but at least they had an author. A person, who sat down and wrote those lines intentionally, someone you could ask why. This docs folder had prompters but no authors. Someone had asked for each file and could explain why they wanted it, but not what was in it or whether it remained accurate. Every file may contain a sensible answer to a sensible request, but no one consciously decided that folder should be over a hundred files deep.
Agents produce verbose volume because volume is the output. At a glance this output can appear reasoned and diligent, so the folder just grows without a decision behind it.
No one reads your 900-line markdown file
Long documents are easy for humans to skim and for agents to consume only in part. When I opened up one of the files the agent had read and used to justify part of the plan, the paragraph that actually mattered was sitting about 400 lines above what I’d been quoted: the approach wasn’t verified and needed further discussion. It had read enough of the file to be confident but not enough to be correct.
I can’t see what my agent reads and skips per file, so this is inference from behaviour rather than something I can prove. What it looked like was sampling.
What’s hard to work with is inherent inconsistency or the non-deterministic behaviour of AI agents. Sometimes files were picked up, other times they weren’t. A source that was authoritative in Tuesday’s session and invisible to the next on Wednesday is worse than a reliable absence. You unintentionally fall into a problem of debugging two systems at once and despite the folder having an index present, it in itself was so long I doubt it was read in whole either.
The issue isn’t necessarily with machine written prose either: an architecture decision record written by a human can fail in the same way and for the same reasons. Length can be its own attribute of failure and no author is exempt from it.
Committing it makes it true
The same team that requires two approvals on a three-line code change will wave through a hundred-line markdown file, because it appears to carry no risk. But it carries a different risk. Code that’s wrong usually ends up breaking things and good tools and observability will tell you. A document that’s wrong can just sit there waiting to be believed.
As far as your agent is concerned, committing the document is what makes it true. A reviewer will skim over a markdown file because it’s not going to introduce any bugs that cause them to stay late on Friday. An agent does the opposite though. It’ll find that file, read it and then treat it as weighted fact because it’s tracked in git and sitting right next to the code that runs. The artefact that received the least human scrutiny ends up with the most machine trust.
Marking the files doesn’t fix it. I considered adding a status header and a last-reviewed date only to realise they were already in the documents. The banner at the top of the document was missed while the agent pulled in the content from the middle. The warning never traveled with what it was meant to qualify. They weren’t being maintained as intended either and I wasn’t going to be able to correct them myself. A draft from the month prior was still marked a draft today, so the label provided nothing.
Treat artefacts as you would code
Every document committed should clear the same bar as your code changes. Who authored it isn’t what matters. An agent-drafted document that a person took time to read, cut down by half and amend can still be a genuinely useful artefact. An entirely human-authored document that won’t ever be read again is still a liability. What differentiates the two is whether anyone judged it before the merge and that’s exactly what your review step is for.
The cost is why you hold that bar at the pull request. Everything in a codebase needs maintaining, tests and tooling included, and a PR is one of the few moments where saying no or requesting changes is cheapest. After that point, you’re either paying to keep the document accurate or having your agents led down the garden path by it. An agent writing the document doesn’t alleviate any of that. It just makes the bill easier to run up without you noticing.
Prefer things that can fail
Most of what you’ll find in long documents is descriptions or details of behaviour and that should be verified rather than lost in prose. If a claim about your system can be asserted, you should be asserting it. A test that stops matching reality will fail loudly and force someone to address it, as opposed to a stale document that sits there until an agent reads it in the coming months.
The job of these documents therefore becomes to do what tools cannot, record decisions. Why did we pick this approach over another, what are the trade offs of this, what did we try first before pivoting. That is knowledge and context traditionally relegated to a wiki, detached from the code and lost shortly after it’s written. Instead, write that knowledge alongside the code and keep it short. Expect it to be read by a person rather than sampled by something.
And the other things? Plans, investigations, spikes, that research that was prompted into existence on Tuesday to answer a question you’d bypassed by Friday? It served its purpose during that task. Delete it when the task ends.
At the end of the day, a human should still decide. Producing and documenting knowledge used to be the expensive part, so accumulating documents felt like progress. Creating documents feels free now, and the scarce thing is human judgement about why something exists and whether it deserves to. That judgement can’t be delegated to the same thing generating the volume.
