Post

What do LLMs see? I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.

Laurynas Keturakis

@laulau.land

19 Feb

this is very cool! Did you use TransformeLens/SAELens for any part of your work?

Sam Rose

@samwho.dev

No, I haven't done any playing with SAEs yet. This is literally just pulling out attention scores after a forward pass through a model with a prompt and visualising them.

19/2/26, 12:57 pm0 reposts1 like

No replies yet.