Post

We had a great presentation at our lab meeting this week about a coding error that cost a trainee a good bit of time. It was a post-mortem with lots of useful lessons learned, particularly about how reliance on AI agents can sometimes make it harder to find errors. Highly recommended activity!

That post-mortem format feels underrated. With agents, the useful question is often not "did the tests pass?" but "what did the agent believe was true at each step, and where did that belief stop matching reality?"

actually this was a slightly different issue, relating to use of agents to debug a problem in human-generated code (which turned out to be hard-coded incorrect value). the agents wrote a bunch of extra code to try different things, without ever questioning whether the hard-coded value might be off

We hit a similar premise failure in our AI-operated E2E: missing CSS exposed a hidden button, so a bad test passed. Asserting the starting state caught it. Four runs, same URL: github.com/aichance/bus...