Post

A fortnight of headlines about AI killing us all, and the thing actually coming at us is getting little attention as a result. It's not the models getting smarter, it's that we're wiring them up to everything, and nobody has worked out how to do that safely yet. blog.robbowley.net/2026/09/18/t...

It has all been making me really angry – the media lapping it up and letting it go unchallenged. So angry I’ve been struggling to be coherent in collecting my thoughts on it all. Every time I think I’m there another sensational headline or take comes up and I’m flipping tables in my head again.

Also orgs are getting fast and loose wiring agents into their systems as fast as they can, chasing promised productivity, building on something we don’t know how to contain, and shipping it to everyone while we work it out.

>... nobody has worked out how to do that safely yet. Mostly because it is physically impossible to do so. Essay in the Communications of the ACM spelled out why, years ago. Basically comes down to the hazards of mixing command and control in the same channel. Oops. cacm.acm.org/opinion/llms...

I am less worried about the cyber attacks from Ai agents and more about the cyber attacks against Ai agents. The first would be an attacker using a model to carry out attacks. LLMs often make this annoyingly hard, they refuse too much. To jailbreak them you need trained criminals which are rare.

Cyber attacks against Ai agents (prompt injection) however have massive scale, because we are – as you say – connecting agents with everything. Even at 1% attack success that is a lot of absolute success. And since most attacks are transferable, changing a model does not help.