Post

Short musings on "cognitive debt" - I'm seeing this in my own work, where excessive unreviewed AI-generated code leads me to lose a firm mental model of what I've built, which then makes it harder to confidently make future decisions simonwillison.net/2026/Feb/15/...

simonwillison.netHow Generative and Agentic AI Shift Concern from Technical Debt to Cognitive DebtThis piece by Margaret-Anne Storey is the best explanation of the term cognitive debt I've seen so far. Cognitive debt, a term gaining traction recently, instead communicates the notion that …

87 reposts459 likes

So eloquently put Simon, as always. I worked in Photography/Advertising for 15 years and stepped away because of whom I was becoming by working in that industry. It taught me to pay attention to the person the work creates, as much as the work the person creates. Context switching radiply=🧠🗑️.

these bots aren't good enough to be let off-leash. you get a big speedup, but not as much as it feels like. and utility is proportional to how much you pay for it

I think the right way to produce, verify, and keep a handle on generated code will be more and more of A Thing over time. That, more than anything, has me viscerally repulsed by "agent teams," Gas Town, other efforts to reduce the human input and review per LoC even more

This looks super interesting, thanks for sharing. I've been writing up super short PRDs in a docs folder of the repo, loosely connected into the original PRD.md. I've found that useful for keeping track of the "why".

A big issue with this is when the agent doesn’t implement what you discussed. You then end up in this situation where a later development relies on something you thought was there, and it turns out it was totally missed. There’s no responsibility on the agent’s side, it’s all on you.

Yeah I have experienced the same. I think we should really spend more time to review and understand AI generated code. I find it helpful to write comments when reviewing the changes.

The debt isn't just in the code — it's in your ability to predict what breaks. You can review every line and still lose the intuition for why certain decisions were made. The fix is probably in the tooling: AI that annotates intent, not just implementation.

I love the "velocity without understanding is not sustainable" line from Mary-Anne's post. Pithy, accurate, and actionable: don't go faster than you can understand.

I've noticed this too. I've noticed that this is actually keeping me somewhat from using agents to their full capability, out of fear that I'd lose my mental model of what I'm (they're) building.

Super useful a have been collecting #cognitivedebt examples for an update on the below, from all kinds of fields but coding particularly good to pay attention to - like speedrunning the social effects we’ll maybe see elsewhere smithery.com/methods/cogn...

smithery.comCognitive Debt“Cognitive Debt is where you forgo the thinking in order just to get the answers, but have no real idea of why the answers are what they are.” Artefacts Newsletter #247, April 25th, 2025 I first wr…

I agree, this is a problem which is easy to run into. But so far, I also found it easy to use AI to dig my way out of this again. I found prompts like "How do subsystems A and B interact?" and "What happens if [some special case] occurs?", with follow-ups, really helpful to get back on track.

I still find it hard to really trust AI with this. It’s great at telling me the intention of systems, but I don’t trust its ability to verify that all such intentions are logically consistent. I think AI is super convinced by its own words, regardless of whether they make sense

Of course, the same happens when humans produce code and design docs. But that’s why we collaborate and review in software — it’s an antidote to mistaken beliefs, and ideally even an antidote to groupthink

So the solution is not through humand being superior, but in humans being less collaborative. And that's why you prompt another AI conversation to review the claims combatively. There is no solidarity amongst the agents.

Another good one is "what tradeoffs did you make when writing this?" I have also prompted AI to write documentation for what it has built (pointing it at great examples of documentation elsewhere) and that has really helped for me.

Once we teach them to write documentation better, it will indeed make these systems more observable. We need to make them talk about the code in a novel way. Write lil O'Reily books about every vibe coded project.

I like to describe this as the race car analogy. AI-based coding is like driving a car at 150MPH, but to get round the track at a decent time, you have to slow down for the corners otherwise you crash. You can let the AI drive when the feature set is a straight road, but then you slow for corners

I've been working on a small-medium app (git says added: 21708, removed: 12239, total: 9469). Even though I've been building it one PR at a time, with intentionality, and reviewing most code, I have found myself surprised by some parts of the codebase "it works like *that*?!"

I could definitely describe the high level architecture accurately, since that's where I've had to direct the LLM the most. But I am often surprised at the lower level details. Despite having read it all. I guess the gulf between doing the thing and reading about the thing is applicable to code too

“no one on the team could explain why certain design decisions had been made or how different parts of the system were supposed to work together” Pre-ai this happened with all software as OGs age out. The “why” of things is lost. Now takes months rather than years.

I'm not worried that changing the implementation or adding new features will become harder. I'm worried that when accidents happen, we no longer have the graceful extensibility and the ability to circumvent the problem to resolve incidents involving several components or sub-systems.

I see nothing in the this new world of more AI-generated code that would reduce surprises or accidents. Everything points to the other direction. The lessons from resilience engineering I think will become more relevant over time.

I recently set up a project in a very "Vibe First" way. Like, mostly I have an agent specify cursor rules, write out goals, designs, etc. Then a bunch of agents go off into git worktrees. I was very concerned by this issue, and my experience is that it "ebbs and flows".

There are times where I'm genuinely like "oh shit idk how that works". But... it hasn't been a problem. When I iterate on the design to push to the next limit, well, I sync with the agent. I have it tell me all about it. That's when I learn how it works, and then I steer it forward again.

There's these very uncomfortable periods where the gap exists but so far it hasn't been a problem, despite this being my most aggressively vibe coded project ever. I'm learning to not feel so nervous, and I'm learning a lot tbh. Those times where I sync up are very interesting.

Notably, I've set things up so that verification is at the absolute core. *Everything* is verifiable. Playwright tests, benchmarks, tons of logs and metrics that the agents all have the ability to call into, pre-commit hooks that force behaviors, etc.

The next edition of the "Mythical Man-Month" should explore the cognitive debt of the "Mythical Agent-Assist." Brook's Law: "Adding manpower to a late software project makes it later" -becomes- "Adding agents to an early software project makes it cognitive harder to maintain/extend/debug"

My conclusion is that you can only go so far not understanding how the project you're building works - you may not need to look at every line of code, but you gotta slow down and make sure your mental model is accurate enough to stay productive

This 100%, but (in these wild times there's always a but) you can ask the robot "how does this codebase work?", followed by "why doesn't it work like the following: ...", and it will happily refactor the entire thing. So I feel that even in 100% vibe mode, you can retake control *if* you want.

I just love thinking about this stuff so much, because no matter what path the discourse takes, you'll always land on "a good engineer will get good results from this". While it feels like everything is changing, things aren't really changing all *that* much.

I am pretty firmly convinced now that we are going to make some of the fastest project start to legacy unsupportable systems ever. Legacy defined as the system is essential but not understood by anyone, cannot be effectively changed. High code generation rates are punishing for teams already.

This is a problem that pre-LLM engineering managers have to deal with right? Since they aren't necessarily hands-on with the codebase, and therefore aren't always up to speed with what was built. How do they deal with it?

Seems like a specific kind of tech debt, no? I’ve noticed this as well when I write AI generated code. It is similar to only partially reviewing a peer’s code - as another user said, LGTM