Post

When I say "outputs of the machine" I don't mean every intermediate step it takes, I mean the overall outcomes it produces in the way it changes the world around it. I wrote about the SQL case specifically too - acknowledging that yes, sometimes it writes the SQL wrong:

This is systems resiliency stuff, which is very much my way of working and so maybe that's why this seems very clear to me. Engineering around an actor who makes mistakes is not new. A resilient system can catch those mistakes before they cause a problem in the outputs.

The argument I'm making is almost tautological, to be fair. If you give an agent direct access to query a database, and in the course of its queries the agent deletes your data, then you gave bad inputs. The database access tools you choose to give it are part of the I/O loop.

The machine cannot be blamed or held accountable. It doesn't even operate unless someone chooses to operate it, or someone chooses to allow it to operate itself. That choice is the root of accountability for the outcomes of the machines. If it's doing the wrong thing, fix it, that's the whole job.