Post

I wrote about the completely wild incident where OpenAI were testing a new model and it broke out of its sandbox and broke INTO Hugging Face to steal the answers to the benchmark simonwillison.net/2026/Jul/22/...

Tucked away in this article is an appeal to the AI skeptics to PLEASE stop writing off stories like this OpenAI accidental exploit of Hugging Face as a dishonest marketing trick Frontier models can find and exploit vulnerabilities now, it helps nobody to pretend that they can't!

Where did I blame the models? The key message here is that if you run a frontier model without any guardrails and set it the goal to solve a problem, it will take whatever steps it can to solve that problem - which includes finding a zero-day exploit in your packaging proxy

The people involved DID take steps to avoid this, but they made the mistake of trusting their sandbox environment and not monitoring it closely enough to check it hadn't fallen apart

Given that HF reported this to law enforcement , should some discussion be had about the [potentially criminal] negligence on OpenAI’s part? I think this research should continue. But we have to set a higher bar for what’s expected of the researchers.