Lukas Muttenthaler

🥳 I am incredibly humbled and grateful to share that our work, "Aligning machine and human visual representations across abstraction levels," has been published today in @nature.com ⬇️

🔮At #ICML2026 we presented Attentive Multi-Layer Fusion for Vision Transformers, an efficient method for decoding information from multiple layers of ViTs for downstream task predictions. It’s almost as performant as full fine-tuning but only an inch more expensive than linear probing!

2/ Wed, Jul 8, 2026 • 10:30 AM – 12:15 PM KST · HALL A #2711 · POSTER Attentive Multi-Layer Fusion for Vision Transformers @lciernik.bsky.social , Marco Morik, @zeynepakata.bsky.social , @lukasmut.bsky.social Paper: arxiv.org/abs/2601.09322 Code: github.com/lciernik/att...

Can context-sensitivity improve human–machine alignment in VFMs? We show that incorporating context improves modeling of human similarity decisions. 🎉Accepted to ReAlign ICLR! Now on @arxiv: arxiv.org/abs/2604.13883 Joint first-authored work with amazing Frieda Born. 1/5

Researcher Spotlight: Dr. Lukas Muttenthaler BIFOLD PhD graduate Dr. Lukas Muttenthaler is pushing AI beyond raw performance, exploring #representational #alignment. More: t1p.de/pe5z5 @lukasmut.bsky.social @eml-munich.bsky.social @tuberlin.bsky.social @mpicbs.bsky.social @xai-berlin.bsky.social

🚨Go check out our most recent preprint on multilayer attention probing for Vision Transformers! It’s almost as performant as full fine-tuning while being similarly compute efficient as standard linear probing! Plus you get interpretable attention maps! More in 🧵👇🏼 Preprint: arxiv.org/abs/2601.09322

Even though I am not part of the Re-Align organizer team anymore, I am still cheering for this workshop! I still (and will always) believe that Re-Align is a great place for connecting the fields of AI / ML, Neuroscience, and Cognitive Science :) Can highly recommend submitting! 🧠🤖

New post! Last week I shared why I thought cognitive (neuro)science hasn’t contributed as much as one might hope to the design of AI systems; this week I'm sharing my thoughts on how methods and principles from these fields *have* been useful in my work. infinitefaculty.substack.com/p/how-cognit...

What aspects of human knowledge do vision models like CLIP fail to capture, and how can we improve them? We suggest models miss key global organization; aligning them makes them more robust. Check out LukasMuttenthaler's work, finally out (in Nature!?) www.nature.com/articles/s41... + our blog! 1/3

We summarize the ideas in a new DeepMind blog post: deepmind.google/blog/teachin... 2/3

When #AI “thinks” like us - New research out now in @nature.com from @lukasmut.bsky.social & @lampinen.bsky.social & @tuberlin.bsky.social & @mpib-berlin.bsky.social forms a bridge between #human and #machine representation. tinyurl.com/nhd4ycsh

Pleased to share that our survey "Getting aligned on representational alignment" — on representational alignment across cognitive (neuro)science and machine learning — is now published in TMLR! openreview.net/forum?id=Hiq... Kudos to @sucholutsky.bsky.social @lukasmut.bsky.social for leading this!

Though it’s traumatic for me to bring this up, I was targeted by a group allied with Turning Point USA in 2018, with the help of Tucker Carlson on Fox. My home address circulated online and I got death threats. 1/

I’m recruiting committee members for the Technical Program Committee at #CCN2026. Please apply if you want to help make submission, review & selection of contributed work (Extended Abstracts & Proceedings) more useful for everyone! 🌐 Helps to have: programming/communications/editorial experience.

Interesting reaction from ChatGPT to the HHS mRNA memo. It finds it so implausible that it thinks it's fake. From the perspective of a ~2024(?) trained model, 2025 policies are so absurd as to be unbelievable... chatgpt.com/share/689364...

🎉 Presenting at #ICML2025 tomorrow! Come and explore how representational similarities behave across datasets :) 📅 Thu Jul 17, 11 AM-1:30 PM PDT 📍 East Exhibition Hall A-B #E-2510 Huge thanks to @lorenzlinhardt.bsky.social, Marco Morik, Jonas Dippel, Simon Kornblith, and @lukasmut.bsky.social!

If two models are more similar to each other than a third on ImageNet, will this hold for medical/satellite images? Our #icml2025 paper analyses how vision model similarities generalize across datasets, the factors that influence them, and their link to downstream task behavior. 🧵1/7

What makes humans similar or different to AI? In a paper out in @natmachintell.nature.com led by @florianmahner.bsky.social & @lukasmut.bsky.social, w/ Umut Güclü, we took a deep look at the factors underlying their representational alignment, with surprising results. www.nature.com/articles/s42...

I am deeply grateful to @lorenzlinhardt.bsky.social, Marco Morik, Jonas Dippel, Simon Kornblith, and @lukasmut.bsky.social for their great work and support in this project! We also thank our collaborators, @bifold.berlin and HFA 7/7 📄Paper: arxiv.org/abs/2411.05561 💻Code: github.com/lciernik/sim...

🚨Have a look at our most recent work on the driving factors of representational similarities! This work will be presented at ICML 2025! See thread below 🧵

Please everyone gather and go outside, protest, march, do something! We cannot let science and academia be destroyed by the worst administration the US has ever had.

How do language models generalize from information they learn in-context vs. via finetuning? In arxiv.org/abs/2505.00661 we show that in-context learning can generalize more flexibly, illustrating key differences in the inductive biases of these modes of learning — and ways to improve finetuning. 1/

That’s a wrap for Re²-Align 🤖 🧠👤 at #ICLR2025 🦁🥳— thanks for being part of it! We’re probably just getting started… see you all at Re³-Align🌷😉

✨ Come join us at the Second edition of the Re-Align workshop @iclr_conf ! 🚀🧠 The workshop explores the fascinating question of how artificial and biological systems align in their representations of the world. #ReAlign #ICLR2025

Harvard has set an example for other higher-ed institutions - rejecting an unlawful and ham-handed attempt to stifle academic freedom, while taking steps to make sure students can benefit from an environment of intellectual inquiry, rigorous debate and mutual respect. Let’s hope others follow suit.

Here’s an extended version of my PhD defense talk about our work on *representational alignment between humans 🧠 and machines 🤖 for computer vision* that I gave two weeks ago at the Max Planck Institute for Human Cognitive and Brain Sciences www.cbs.mpg.de/cbs-coconut/...

🚨 We're hiring! If you're excited about 🤖 ML/LLMs, 🧠 cognitive science, or 💭 computational psychiatry, come join us in Munich. Two fully funded PhDs @www.helmholtz-munich.de: tailored mentorship, international vibe, lots of room to grow. 📅 May 16th 🔗 hcai-munich.com/PhDHCAI.pdf

Thank you for having me!

Join us for the talk by @lukasmut.bsky.social (Google Deep Mind, TU Berlin) on "Representational alignment of humans and machines for computer vision" Starting soon at 10 AM (CET), March 28 Find the Zoom Link here: www.cbs.mpg.de/en/cbs-coconut

Training objective drives the consistency of representational similarity across datasets Laure Ciernik, Lorenz Linhardt, …, Simon Kornblith, Lukas Muttenthaler @lukasmut.bsky.social 👉 Indeed, people need to better appreciate the interaction of datasets and hypotheses arxiv.org/abs/2411.05561

I encourage all of my friends and fellow scientists in the US to go outside and march against Trump’s and Elon’s oligarchy. There’s nothing at this point that is more important than fighting for democracy! Democracy is in real danger and we here in Europa are concerned about the recent developments.

Now almost 450 signatories to my open letter calling on the @royalsociety.org to stand up for its values & deal with the widespread concerns raised by Elon Musk's Fellowship. Please consider signing & sharing: forms.gle/miDciq35oxyw... List of signatories here: occamstypewriter.org/scurry/2025/...

Our representational alignment workshop returns to #ICLR2025! Submit your work on how ML/cogsci/neuro systems represent the world & what shapes these representations 💭🧠🤖 w/ @thisismyhat.bsky.social @dotadotadota.bsky.social, @sucholutsky.bsky.social @lukasmut.bsky.social @siddsuresh97.bsky.social

Re-Align returns to #ICLR2025! Please consider submitting if you work on how ML / CogSci / Neuro systems represent the world & what factors shape these representations 💭🧠🤖 w/ @thisismyhat.bsky.social @dotadotadota.bsky.social @sucholutsky.bsky.social @eringrant.bsky.social @siddsuresh97.bsky.social

This is an incredibly moving and powerful read. We shall see it as a reminder to check in with the people around us and help build a collaborative culture of kind relationships that encourages openness about our mental states rather than a competitive culture full of selfishness and greed.

Personal vision tasks–like detecting *your mug*--are hard; they’re data scarce and fine-grained. In our new paper, we show you can adapt general-purpose vision models to these tasks from just three photos! 📝: arxiv.org/abs/2412.16156 💻: github.com/ssundaram21/... (1/n)

What counts as in-context learning (ICL)? Typically, you might think of it as learning a task from a few examples. However, we’ve just written a perspective (arxiv.org/abs/2412.03782) suggesting interpreting a much broader spectrum of behaviors as ICL! Quick summary thread: 1/7

Unfortunately, I am not attending NeurIPS this year but don’t miss our work on perceptual alignment presented by @shobsund.bsky.social and Stephanie Fu in the Thursday morning session! 🧠🤖

Excited that Re-Align will have its second iteration at ICLR 2025 in Singapore! More soon! 🧠🤖

Reposting this because I just started using Bluesky 🦋 Excited to share what we’ve been working on during my time at DeepMind! We came up with an approach that can distill the hierarchical structure of human conceptual knowledge into vision foundation models via a surrogate teacher model. More below👇🏼

🚨 We just updated our perspective on representational alignment. The most recent version is both more crisp and more comprehensive. We try to find common language across research disciplines for aligning the representations from different info processing systems! arxiv.org/abs/2310.13018

We employed a triplet odd-one-out task (🥸,🤓,😎) and distributed it online to thousands of participants. We then trained sparse (#VICE, @lukasmut.bsky.social) and deep encoding models (#VGG-Face) to investigate the latent representations underlying human face similarity judgments. #alignment (2/n)

We think this work shows an interesting perspective on what our models may be missing and how to address it using representational alignment — excited to hear your thoughts! Check out the preprint here: arxiv.org/abs/2409.06509 9/