Post

Larry Osterman
@larryosterman.github.io

Can someone help me understand how an LLM can hack into a web site? Fundamentally an LLM is a chat bot. It takes text in and spits text out. The text out can be an instruction to a host to create a file or fetch a url, or even to write a program.

Larry Osterman
@larryosterman.github.io

But ultimately everything done by the LLM is an action performed by the host. And the host can and should have guardrails. And the host can be in a vm, with egress filters, firewall rules, etc. Is the claim that the llm first broke out of the sandbox it was in? If so, that seems like a big deal.

0 reposts1 like

Larry Osterman
@larryosterman.github.io

I'm also confused about the LLM ignoring the denial of a request. Ultimately the host either performs the actions requested by the LLM or it fails the request to perform the action. The host doesn't ever say 'no, I don't want you to do that' - it just fails the operation.

🌍 Duncan Kimpton 🛠️
@dgkimpton.bsky.social

It's a loop basically, the "agent" program takes a prompt like "solve this problem" and asks the llm how to do it, then does what it asks, gives it the output, asks what next, does it, around and around until the problem is solved. The only human interaction was the first step defining the problem.

Steve Syfuhs
@syfuhs.net

There are two fundamental problems. The first is that most guardrails are instructions to the LLM "don't do X". That's because problem two is that the sandbox is pretty dumb about what is safe and isn't safe which is how you end up with /yolo or /allow-all otherwise it's gated on a human ACKing.

Larry Osterman
@larryosterman.github.io

Doesn't that require you running hacking experiments without any sandboxes or at least monitoring? That's the part that confuses me. How do you run experiments without any monitoring?

Steve Syfuhs
@syfuhs.net

The sandboxes are getting better but the LLM is trained to get the problem solved and "don't do X" is more of a suggestion so it's this ratcheting problem. What might look like a perfectly safe operation to the sandbox is just one tool in the solution to the problem.