Post

Google have put a classifier inside the Gemini 4 Argon API that injects some dynamic context right under the developer instructions, mostly as an anti-'jailbreak' thing, but it also fires for at least one perfectly innocent reason (if you ask it its model version).

Boxo McFoxo

@boxobark.ing

They have an injected turn role for dynamic context (called, appropriately, `context`) that goes at the bottom of the context window right above the latest user message, so that it doesn't clear out the entire KV cache.

23/10/26, 3:47 am0 reposts1 like

Boxo McFoxo

@boxobark.ing

But they believe their own mythos about the model actually understanding roles, so they made it a second `system` turn. Meaning the KV cache gets cleared out twice, first when the classifier inserts something, and a second time, when it stops inserting it.

Boxo McFoxo

@boxobark.ing

Pretty hostile product design considering cached tokens cost 95% less. Though an LLM trained to appear as if it is a thinking, understanding, conversing entity is already hostile and unethical product design anyway, of course!