Post

A fortnight of headlines about AI killing us all, and the thing actually coming at us is getting little attention as a result. It's not the models getting smarter, it's that we're wiring them up to everything, and nobody has worked out how to do that safely yet. blog.robbowley.net/2026/09/18/t...

It has all been making me really angry – the media lapping it up and letting it go unchallenged. So angry I’ve been struggling to be coherent in collecting my thoughts on it all. Every time I think I’m there another sensational headline or take comes up and I’m flipping tables in my head again.

At least it's out of my head now, but I'm genuinely worried, particularly for the tsunami of cyber attacks I see coming

>... nobody has worked out how to do that safely yet. Mostly because it is physically impossible to do so. Essay in the Communications of the ACM spelled out why, years ago. Basically comes down to the hazards of mixing command and control in the same channel. Oops. cacm.acm.org/opinion/llms...

I am less worried about the cyber attacks from Ai agents and more about the cyber attacks against Ai agents. The first would be an attacker using a model to carry out attacks. LLMs often make this annoyingly hard, they refuse too much. To jailbreak them you need trained criminals which are rare.

Cyber attacks against Ai agents (prompt injection) however have massive scale, because we are – as you say – connecting agents with everything. Even at 1% attack success that is a lot of absolute success. And since most attacks are transferable, changing a model does not help.

A lot of people in the industry are thinking about this though. I held a talk yesterday that „LLMs are not Skynet“ and the echo was that I focused on disproving the wrong thing (LLMs are not conscious) and should have focused on what you wrote in your blog.