Post

What do LLMs see? I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.

So the opacity is a running average score?

What do you mean?

You wrote “average attention score as the prompt progresses” so I was just clarifying for myself average over what