Post

This is why i believe multi agent / steps are the most effective for LLM consistency. You have one agent talking to the use and understanding the request. Another agent to run the tools get the data. Then a final agent to send the response back to the user. Trust nothing. Every prompt is an attack

The Dual LLM pattern I proposed a couple of years ago works something like that: simonwillison.net/2023/Apr/25/...

I really feel like we are solving “sql injection” all over again. Sanitize the inputs…

Have you considered expanding this to three agents? A: Fetches articles, can't read them B: Reads/summarizes, zero external access C: Outputs results, never sees original content B must justify requests to both A ("why do you need these emails?") and C ("why send this data here?").

And yes, this is effectively me trying to get the world to care about prompt injection by trying out a new term for a subset of the problem! I hope this captures the risk to end users in a more visceral way - particularly important now that people are mixing and matching MCP servers themselves

I'm curious what "prepared statements" will eventually look like in LLM world. Having an agent check for injections in another agent's output feels more like the equivalent of sophistically checking the output of a "normal" SQL query, no? Will we need new LLM architectures to fully eliminate it?

That's still the million dollar question - we have a version of that now in the separation of system prompts from regular prompts but it's leaky - the regular prompts can still override the system prompt if they try hard enough

And they will have to. You could not read a "chose your own adventure book" otherwise. Following instructions as they come up is a natural behavior of what agents have to do.

This is a beautiful way of describing the constant communication I have with our AI dept. Mind if I steal this phrase?