Post

Simon Willison
@simonwillison.net

OpenAI's CISO Dane Stuckey posted an essay (on Twitter) about how their new ChatGPT Atlas browser attempts to deal with the risk of prompt injection attacks, I ended up writing a point-by-point commentary on my blog: simonwillison.net/2025/Oct/22/...

simonwillison.netDane Stuckey (OpenAI CISO) on prompt injection risks for ChatGPT AtlasMy biggest complaint about the launch of the ChatGPT Atlas browser the other day was the lack of details on how OpenAI are addressing prompt injection attacks. The launch post …

12 reposts39 likes

Ein Realist
@einrealist.bsky.social

“OpenAI have an advantage here of being a centralized system—they can monitor their entire user base for signs of new attack patterns.” How do they detect this? And what data do they need to collect to detect this? (Sorry for beating the privacy drum again.) These AI browsers are a 'No' for me.

Simon Willison
@simonwillison.net

I don't know and I would love to find out Best case scenario is they are using privacy-preserving analytical techniques like Claude's Clio, but right now we have no reason to believe they are doing that since they haven't shared any details simonwillison.net/2024/Dec/12/...

simonwillison.netClio: A system for privacy-preserving insights into real-world AI useNew research from Anthropic, describing a system they built called Clio - for Claude insights and observations - which attempts to provide insights into how Claude is being used by …
Ein Realist
@einrealist.bsky.social

Interesting. Yes, I can see how this kind of 'destructuring' and anonymization could help. At the very least, it would enable outlier detection. However, it also removes a lot of context, making it difficult to analyze actual attacks. You would need to somehow measure its effectiveness.

Buck Fillionaires
@buck-fillionaires.bsky.social

The prospect/risk of “AI hacking” by prompt is a whole new can of worms. I don’t see any advantage of integrating AI into the browser as it’s another vulnerability/target. If I want AI in the browser, I’ll point the URL bar to a cloud AI service.