Post

I'm reviewing all agent code for client projects and having good success lately. Very few bugs or defects. If we're reworking things, it's because I didn't spec clearly enough or changed my mind after seeing the code. But, I've had one area of repeated failure in recent weeks…

24/7/26, 3:54 pm0 reposts6 likes

Jeremy Smith

@jeremysmith.co

24 July

And that's on a project where I'm slowly migrating complex authorization rules from CanCanCan to ActionPolicy. During review, I've extracted relevant rules from both sides and walked the authz logic myself, and in this way I've caught multiple mistakes in the translation of rules.

Jeremy Smith

@jeremysmith.co

24 July

Now I'm stopping to build a better migration process (including a set of comprehensive bridge tests). Just thought it was a good data point where code has the appearance of correctness (easy to just LGTM), test coverage is higher, CI is green, and you're still in a failure mode.

Casper Klenz-Kitenge

@cabgfx.com

24 July

This is expert level freeze frame, seems like a super teachable moment 🔬🔬🔬

Jeremy NT

@jeremynt.bsky.social

24 July

This is a good cautionary tale because it's something so critical... did you use an agent to review the PR first and if so did it also miss the issues?

Jeremy Smith

@jeremysmith.co

24 July

Yes, my typical process has been to have another agent review the PR. And for this, I've used 2 different reviewing models. This project has been intermittent over the past few weeks. But from what I can remember, reviewing agents might have caught a couple issues, but I've manually caught 3-5 more.