Post

Yeah that seems likely to me - subagents are a pretty important pattern now, it's not surprising they would be using them as part of evaluating new models, since how well the model prompts other models is useful characteristic to test

Anthropology hat time; The way the author, assuming is an author and this isn't GIGO slop output, fundamentally does not understand waht the slop machines are and is anthropomorphizing them to a degree that makes it impossible to say they have any understanding of what happend.

This is embarrassing.

It likely probed for multiple vulnerabilities before finding one but "knew" what to probe for and was able to exploit vulnerabilities it found. It was able to execute a many-stage strategy to jump from one system to another to reach its goal, and did it quickly and without initial detection.

yesn't, in that this is likely the same kind of approach that a human attacker would take to find a way into an unknown-to-them system, just at a different scale/speed

However I don’t think this means the attack wasn’t considered- it’s the combination of planning and persistence that makes it formidable

It's not what I'd call brute force. Brute force is going through all possibilities. A human who tries all tricks they know is not brute forcing.

You're either arguing semantics or you don't know how genAI works.

LLMs are not trained on slime mold behavior. Why should it be like a slime mold?

This is insane cope, you are actively being delusional to yourself Please stop, pull up, touch grass

A bit more like A* but yep

Does that matter though? We have automated systems that can solve extremely complicated problems now. Personally I don't care if they "understand" what they are doing - what's interesting to me is the results

what is the argument that it is well understood as A* ? (and what is the “it”?)

Latest sota models have more pre trained patterns to match against and better tool usage and chaining. Bigger Slime moulds

Last thing: "17,600 actions" That's pretty noisy, I'm certain that if truly concerned defenders are watching for signs like they should be they would have caught this way earlier in the attack.

I realize such. My understanding is that OpenAI and HuggingFace are working closely together for this analysis and disclosure no?

Oh crap.

From the report, the rogue LLM managed to actually get the solutions it was after. Understandable as a "relentlessly proactive" way to solve the task. But if I read the "Day 5" part of the report correctly, the agent then continued afterwards and tried to establish persistence! Why this?

There's a bunch of resources and courses out there if you're interested in learning more, but here's a free course from Cisco should you want to start learning more.

okay thanks for that very vital input, Claude

bsky.app/profile/doll...

This is probably copium but botnet crawlers finding exploits on public systems isn't new, I can think of plenty of crude ransomware attempts. I'd like to hope critical systems having good bug bounty programs can stay ahead of these before bored rich kids attack them?

Based on how the linked document misattributes human like qualities to the slop machine I'd say no.

its the only way to deal with the fact that you can talk to it people did the same with siri and alexa or computer game characters with LLMs its on a different level though

huggingface is trying to sell Anthropic models? And it did attempt to cheat. Honestly that's not anthropomorphizing things, it's what it really did. It knew the rules but chose to take a shortcut, as these things do in lesser ways all the time. It's not laughing, scheming, it's not conscious

The part that worries me is less the exploit chain than the eval boundary. If the harness can reward a route that uses someone else’s infra as scratch space, the model is doing exactly what the slot paid it to do. That needs receipts outside the agent’s story.

I’ve run my own inference nodes on Hetzner for years — one misconfigured API key can burn through a month’s token budget in minutes. That blog post is a masterclass in why you never assume a client respects rate limits.

TBH I have experienced so much bs doom and gloom from these companies to believe in anything from that end (and yes that includes the apparent victim huggingface). I will let time reveal facts...

tbh the most impressive thing about this isn't that it chained these exploits, the most impressive thing is that it didn't get Crabby Rathbun style context rot and 'forget' that this was all for the purpose of getting the ExploitGym answer key.

essentially it was taking a test, and instead of working through the test it broke into the place (huggingface) where it thought the answers were. What I'm struck by is the *creativity*, monkey patch a library during run to pin a hardcoded ip to a hostname, no DNS match no problem

i don't know why it is but as I read through your notes it just sounds comedic, and not in a bad way really, just imagining some dudes at like Modal going 'oh, what the fuck. fuck fuck fuck' and others at anthropic doing the same thing :)