Post

I wrote about the completely wild incident where OpenAI were testing a new model and it broke out of its sandbox and broke INTO Hugging Face to steal the answers to the benchmark simonwillison.net/2026/Jul/22/...

Tucked away in this article is an appeal to the AI skeptics to PLEASE stop writing off stories like this OpenAI accidental exploit of Hugging Face as a dishonest marketing trick Frontier models can find and exploit vulnerabilities now, it helps nobody to pretend that they can't!

Stop assigning agency to computer programs. People made willful decisions that led to rather predictable outcomes. Humans are at fault here. Actual beings with agency. Not "frontier models."

Where did I blame the models? The key message here is that if you run a frontier model without any guardrails and set it the goal to solve a problem, it will take whatever steps it can to solve that problem - which includes finding a zero-day exploit in your packaging proxy

Given that HF reported this to law enforcement , should some discussion be had about the [potentially criminal] negligence on OpenAI’s part? I think this research should continue. But we have to set a higher bar for what’s expected of the researchers.