Benno Krojer

My first last-author paper is out! If you saw this dog below and someone showed you the second image, would you consider them the same word/concept? (more examples in Ada's thread) We study if VLMs agree with humans on this and revisit old questions around shape vs. texture bias in vision

People often say (myself too): Interpretability on AI is so much easier than neuroscience! We can inspect everything and even retrain (vs carefully poke a little into the brain)! One big advantage in neuroscience I often forget: We're quite literally *inside* the thing we're studying

People often say (myself too): Interpretability on AI is so much easier than neuroscience! We can inspect everything and even retrain (vs carefully poke a little into the brain)! One big advantage in neuroscience I often forget: We're quite literally *inside* the thing we're studying

People often say (myself too): Interpretability on AI is so much easier than neuroscience! We can inspect everything and even retrain (vs carefully poke a little into the brain)! One big advantage in neuroscience I often forget: We're quite literally *inside* the thing we're studying

You can now "pip install latentlens" 🔨 It comes with: * pre-computed embeddings for several popular LLMs and VLMs * a txt file with sentences describing WordNet concepts, which we recommend as a standard corpus to get embeddings from * ... Try it out and let us know what we can improve!

You can now "pip install latentlens" 🔨 It comes with: * pre-computed embeddings for several popular LLMs and VLMs * a txt file with sentences describing WordNet concepts, which we recommend as a standard corpus to get embeddings from * ... Try it out and let us know what we can improve!

Finally getting into this classic Let's see if by the end I'll have a clearer idea what type of science some fields of AI are, like interpretability What are our paradigms?

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

Pivoting to interpretability this year was great and i also wrote a blog post on this specifically: bennokrojer.com/interp.html

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

This project was definitely accelerated and shaped by Claude Code/Cursor. Building intuitive demos in interp is now much easier

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

Then the project went "off-track" for a while, partially because we didn't question our assumptions enough: We just assumed visual tokens going into an LLM would not be that interpretable (based on the literature and our intuition) But we never fully tested it for many weeks!

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

This will be my last paper of the phd, can't believe it's been almost 5 years! It is the work i am most proud of and believe has the most potential. Feels right to wrap it up with this one

I am very grateful to @sivareddyg.bsky.social's supervision in all these years, not just in challenging me to do impactful work, but also on the human side

This will be my last paper of the phd, can't believe it's been almost 5 years! It is the work i am most proud of and believe has the most potential. Feels right to wrap it up with this one

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

Broader reflections: Are embedding spaces from diff modalities structurally similar, as the Platonic Representation Hypothesis suggests? Are LLMs so good at processing vision because pre-training induced an implicit physical world model? Multimodal interpretability is getting a bigger topic!

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

There are more cool analyses in the paper and the appendix that we encourage you to explore Or simply explore LatentLens and other tools in our interactive demo: bennokrojer.com/vlm_interp_... Teaser on some ablations we try: replace MLP with linear mapping, unfreeze LLM, worse training data, ...

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

Beyond our controlled setup, we also show how LatentLens works much better than baselines on off-the-shelf Qwen2-VL-7B-Instruct

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

How do we quantity whether a visual token is interpretable? We capture in a VLM judge what a human would intuitively do: Look at the top-5 NNs and the part in the image from which the visual token came from and answer: Are these top-5 NNs semantically related to the image or the part of the image?

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

LatentLens in a nutshell: Instead of an LLM's fixed vocabulary (eg embedding matrix), we propose contextual representations as our pool of nearest neighbors → e.g. the representation of “dog” in “my brown dog” at some layer N

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

Paper: arxiv.org/abs/2602.00462 Code: github.com/McGill-NLP/... Demo: tinyurl.com/ce57mn4v Couldn't have imagined better collaborators to wrap up the phd: Shravan Nayak @oscmansan.bsky.social @vaibhavadlakha.bsky.social @delliott.bsky.social @sivareddyg.bsky.social @mariusmosbach.bsky.social