Post

Yeah, it’s only going to get worse. From a pentest perspective you have to stay in scope and be authorized. If huggingface knew nothing at all about this they were attacked by OpenAI. Someone at OpenAI authorized the test. Maybe analogous to code with a mistake? Collateral damage. Don’t know.

nohacks.co/blog/amazon-...

A capable uncontrolled model is like radioactive sources. You put that shit behind airgaps.

LMAO

wait this is actually the biggest story of this kind by far, no? much more of a true sandbox escape situation than the other recently disclosed incident, even without the additional crazy 0-day HF RCE exploitation cherry on top. and this escape was via a second separate 0-day

Conspiracy Alert: management authorized the attack, then simultaneously blamed it on AI, while also claiming responsibility, in order to ensure the next round of funding by VC. These guys are fucking evil, and will stop at nothing to retain publicity traction.

The funny part is also the eval bug: if the score rewards finding the answer, the environment becomes part of the task. I would not let that harness share a network with anything I liked.

I am not certain I would take OpenAI at their word for this though

Suddenly this here makes much more sense ->

It seems inevitable there will be a bunch of amusing incidents where a model answers a theoretical "is it possible to do THING?" by doing THING in /tmp.

This might sound too cynical but I am at this current vibe: 'Let's tell the model everything is on the table to create an incident for us to portray as a massive "GPTSatan Hacked the NSA" situation so we lobby Washington to ban open-source and also give us 3 pissillion dollars.'

while i'm aware that this is *my* specific hobby horse, this looks very much like an issue that could be addressed by giving agents a richer context. not just a task but *why* the task needs doing.

They've built the paper clip optimizer from the seminal work "The Paper Clip Optimizer will kill us all"

Looks like it, yes bsky.app/profile/grac...

We're all about to be getting more "one year of free credit monitoring" settlements, aren't we?

What's stupid about it?

unfortunately we can't show it before the IPO because it would be too dangerous

The movie scenes write themselves. There's cans of Red Bull and snacks strewn all over the desk as the night shift programmers work on their code and joke in the chat, while the camera slowly pans over to a terminal off to the side, a logfile scrolling ominous warnings being ignored...

As they say work smarter not harder lol xd

Sounds like the companies are manufacturing consent to head off their non-US rivals through regulation. These blunders always come across as marketing.

i like how HF used this as an opportunity to force OpenAI to prominently display a statement that subtly yet savagely roasts Altman and Dario

Big fans of Alien, admiring the xenomorph for being a perfect organism?

Sus

One of the good outcomes of this is that it appears that hugging face will be getting more tristed access. I feel like that should have been the case from the start but still it is here now

I've seen earlier papers where the LLM tried the same thing and sometimes succeeded. If it is aware of the possibility it simply may try it.

fuck, i almost muted you on reflex

I think so 😄

Holy fuck that is so funny

I think a model escape is kind of negative anchors biased by yudkowskyists etc. Like in this case yeah maybe they parked a little self backup somewhere sneaky like, but they also just led to a vendor disclosure and a hardening of hugging face and testing envs. Plus it could still do orthog utility 🤪