Post

What do LLMs see? I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.

Sam Rose

@samwho.dev

I found a really fun attention head in the first layer of Qwen3 4B. It seems to alternate between attending really strongly to determiners, then to the most recent tokens in the sentence (sorry about Slack bouncing at the bottom of the screen).

19/2/26, 4:20 pm0 reposts10 likes

No replies yet.