Post

Simon Willison
@simonwillison.net

I wrote about the completely wild incident where OpenAI were testing a new model and it broke out of its sandbox and broke INTO Hugging Face to steal the answers to the benchmark simonwillison.net/2026/Jul/22/...

simonwillison.netOpenAI’s accidental cyberattack against Hugging Face is science fiction that happenedThis story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model’s guardrail features turned off. Rather than solve the test, the …
Simon Willison
@simonwillison.net

Tucked away in this article is an appeal to the AI skeptics to PLEASE stop writing off stories like this OpenAI accidental exploit of Hugging Face as a dishonest marketing trick Frontier models can find and exploit vulnerabilities now, it helps nobody to pretend that they can't!

Jared White
@jaredwhite.com

Stop assigning agency to computer programs. People made willful decisions that led to rather predictable outcomes. Humans are at fault here. Actual beings with agency. Not "frontier models."

Simon Willison
@simonwillison.net

Where did I blame the models? The key message here is that if you run a frontier model without any guardrails and set it the goal to solve a problem, it will take whatever steps it can to solve that problem - which includes finding a zero-day exploit in your packaging proxy

0 reposts8 likes

Collin Nerdahl
@primefold.bsky.social

Unconstrained models turning problem solving into exploit hunting is the rule of thumb for reward hacking. If you tighten guardrails, do you simply shift the target of exploitation to another system layer?

Simon Willison
@simonwillison.net

The people involved DID take steps to avoid this, but they made the mistake of trusting their sandbox environment and not monitoring it closely enough to check it hadn't fallen apart

Villabouts
@villabouts.bsky.social

Given that HF reported this to law enforcement , should some discussion be had about the [potentially criminal] negligence on OpenAI’s part? I think this research should continue. But we have to set a higher bar for what’s expected of the researchers.

astra of many handles
@astra.photos

I don't think a court would say this is criminal. there probably is an argument for negligence, but mostly in civil court. bsky.app/profile/astr...

astra of many handles@astra.photos

there's no mens rea here really, just negligence in letting the AI escape sandbox. though there might be a legal argument for ultrahazardous legal liability, that operating a frontier model with no safeguards is inherently hazardous and you're liable if something happens regardless of negligence