Post

What do LLMs see? I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.

maybe: leif weatherby

@leifw.bsky.social

19 Feb

do you have this up somwhere? i would love to teach with this, or show it to people at talks (with credit) so they can quickly grasp this point

Sam Rose

@samwho.dev

Just put the latest version here: gist.github.com/samwho/6e351... It's messy and vibey and I've only tested it with 2 models so it's possible that different attention schemes confuse it. Looking at specific attention heads near the start of the model (top of grid) is most interesting ime.

19/2/26, 5:32 pm0 reposts2 likes

maybe: leif weatherby

@leifw.bsky.social

19 Feb

Would be extremely interesting to formalize sequences, but literacy wise this solves like 25 problems in 10 seconds this is god’s work