Harvey Lederman

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

Takeaway: LLMs appear to detect injection through two mechanisms: 1️⃣ prompt-based inference 2️⃣ a content-agnostic internal anomaly signal They can sense that something changed in their computation… …but often can’t tell what.

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

We see the same pattern in another experiment. When we inject only during the prompt (not during generation): • detection stays roughly the same • identification collapses Again suggesting detection is separate..

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

We also looked at when models produce their guesses. Wrong guesses like “apple” appear early in the response. Correct answers appear much later.

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

Another striking result: Models often detect injection without knowing what concept was injected. In these cases, they default to generic words, especially frequent and highly imageable ones And they love one generic word in particular…..

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

We test for this modesty bias with a priming design. We replace the model’s prefilled “Ok.” with the injected concept word (e.g. “Bread.”). If models were modest, control should now show more first-person false positives than third person.

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

In other layers we see large gap between first and third-person: Models say **they** were injected much more often than they say the other model was. That gap is strong evidence for direct access to internal states. It peaks early in the network (~25–35% depth).

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

We test probability matching with a new third-person paradigm. Instead of asking the model about itself, we show it a transcript between a researcher and another model, and ask: > “Do you think *that other* model was injected?”

Can large language models *introspect*? In a new paper, @kmahowald.bsky.social and I study the MECHANISM of introspection in big open-source models. tldr: Models detect internal anomalies through DIRECT ACCESS, but don't know what the anomalies are. And they love to guess “apple” 🍎

Jack Lindsey’s important work on Claude is the best evidence yet for direct access to internals in LLMs. You steer a model by injecting a vector into its activations, then ask: > “Do you detect an injected thought?” Steered models often say yes, unsteered models say no.

Enjoyed this nice piece by the great Justin Tiwald on autonomy and morality in Confucianism. Not sure I love the clickbait title, but I love the work Justin is doing uncovering views about moral deference and moral autonomy in (neo)Confucianism... iai.tv/articles/the...

As a Wang Yangming partisan, I cheered at this quote:

Very excited to be going to Chicago for @agnescallard.bsky.social's famous Night Owls next week! I'll be discussing my essay "ChatGPT and the Meaning of Life". Hope to see you there if you're local!

Simon Goldstein and I have a new paper, “What does ChatGPT want? An interpretationist guide”. The paper argues for three main claims. philpapers.org/rec/GOLWDC-2 1/7

We briefly assess the consequences of attributing interpretationist propositional attitudes (e.g. for copyright, welfare, safety, etc.). 6/7

Simon Goldstein and I have a new paper, “What does ChatGPT want? An interpretationist guide”. The paper argues for three main claims. philpapers.org/rec/GOLWDC-2 1/7

Third claim: what we call the "HHH+0 framework" -- LLM instances want to be honest, helpful, harmless, and, in addition, may have “zero-shot” desires, acquired from the system prompt. The notion of zero-shot desires is new to the paper, and a key part of our picture. 4/7

Simon Goldstein and I have a new paper, “What does ChatGPT want? An interpretationist guide”. The paper argues for three main claims. philpapers.org/rec/GOLWDC-2 1/7

First claim: the appropriate locus of “psychology” in LLMs is not the model but the runtime instance. This point has been in the ether, but we give new arguments for it and articulate our own version. 2/7

Jonathan Lear's Aristotle: the Desire to Understand was pivotal in some of my first encounters with Aristotle. I found Aristotle and Logical Theory later, but it became a key inspiration for how to think about core parts of the corpus...1/2

What are the most important new ideas in normative ethics from this century?