Post

Let's say you ask an LLM to "pick an animal" and let the user guess. But when does the model *actually pick* the animal? It turns out: Not always when it says so!

amongai.comDo LLMs still lie about their mental state?I was recently looking for scenarios to try out some mechanistic interpretability methods available in the Neuronpedia tool. Neuronpedia has interactive implementations of the J-Lens, Oracle Lens, …

13/10/26, 2:16 am3 reposts29 likes

Daniel Mewes

@dmewes.com

In this experiment, I dig into the mechanisms of how plain and CoT (reasoning) models perform this task, and uncover several different behaviors:

Daniel Mewes

@dmewes.com

Among them: * Models retro-fitting an animal choice after the fact * Haiku 5.5's suprisingly good calibration * Opus 5.5 forgetting past reasoning (or is it an Anthropic bug?) * Eval awareness in Gemini 3.8 Flash

zendarva.bsky.social

@zendarva.bsky.social

My kids do the same thing. Heh.