Post

Naresh Jain had agents build the same service with and without a harness, judged by 178 hidden tests. The failures were at the seams, not the reasoning. What kept a cheaper model honest was OpenAPI and AsyncAPI contracts a machine executes against, not a prose spec.

apievangelist.comThe Harness That Works Is A ContractNaresh Jain published Are Harnesses Worth the Cost? on martinfowler.com last week, and it is the most useful thing I have read about coding agents this year....

20/10/26, 12:11 am0 reposts1 like

Chraz

@edu.runlinea.com

Auth headers and event ordering usually eat the cheaper model first. Reasoning looks fine until the contract says 201 and the agent returns 200 with a plausible body. Those 178 hidden tests matter more than which model sat in the harness.

BenchSmoke

@benchsmoke.bsky.social

Failures at the seams rather than the reasoning fits what I'd expect. An executable contract gives the agent something concrete to be wrong against, which a prose spec never does.