Kathy Garcia

biorxiv.orgTranslating functional brain activity between humans and monkeysComparing brain responses across species is essential for identifying models of human cognition and testing theories of brain evolution, but differences in brain size, measurement techniques, and sign...

What dimensions organize how we see social interactions? We learn them from 53,000+ human judgments of naturalistic social clips. They explain 2× the variance of pre-labeled annotations & better predict visual cortex, EBA, and pSTS New preprint w @lisik.bsky.social 🧵 osf.io/preprints/ps...

Kathy Garcia reposted

Chris Baker

@cibaker.bsky.social

A new method for translating functional brain responses between species - Spatiotemporal Hyperalignment projects shared information into a common representational space Led by @kbraunlich.bsky.social Marianne Duyck with @bevilconway.bsky.social www.biorxiv.org/content/10.6...

Kathy Garcia reposted

arxiv.orgSimple 3D Pose Features Support Human and Machine Social Scene UnderstandingHumans can quickly and effortlessly extract a variety of information about others' social interactions from visual input, ranging from visuospatial cues like whether two people are facing each other t...

23h

Excited about this new work led by @gkathy.bsky.social: Human similarity judgments reveal fine-grained representations of dynamic social scenes osf.io/preprints/ps... 🧵👇

Kathy Garcia@gkathy.bsky.social

Kathy Garcia reposted

Wenshuo Qin

@qinwenshuo.bsky.social

10 Nov 2025

Why do AI models struggle with social scenes? 🧐 Our new preprint with @lisik.bsky.social reveals a fundamental gap: most AI vision models lack explicit 3D pose information that humans rely on for social judgments. Read the full work: arxiv.org/abs/2511.03988

📄 Preprint: osf.io/preprints/ps... 💾 Data + code: github.com/garciakathy/... 🔎 Interactive project page: garciakathy.github.io/latent_dimen... Questions and comments are welcome!

arxiv.orgBehavioral Geometric Supervision Aligns Video Foundation Models with Human Social PerceptionCurrent video foundation models, including the strongest self-supervised models such as V-JEPA2, fail to capture how humans organize social information in dynamic scenes. For example, across a range o...

Related: we these human judgments to fine-tune video foundation models toward human social perception in a complementary paper: arxiv.org/abs/2510.01502

Interestingly, these same learned dimensions do a much better job at predicting fMRI neural responses (N=4; McMahon et al 2023) in the every participant, including EBA, ventral temporal cortex, and pSTS

Takeaway: People seem to represent social scenes as fine-grained representations that prior accounts might not fully capture And social-affective information may matter more in lateral "visual" cortex than prior action-based accounts assume

The top 11 dimensions are interpretable and stable across 20 retrained models. Many combine who is involved with what they're doing: conversation, caregiving, infant play, cooking & eating, children dancing. Labels: contrastive analysis of human-written captions.

We also find broad semantic categories split: Infant caregiving and infant play involve the same kinds of people and settings; the 13 annotations describe them near-identically Yet judgments split them into separate dimensions (only 3 of their top-20 videos overlap)

For the behavioral task, we had over 232 people watch 3 sec clips of two-person interactions and pick the odd one out. Which would you pick? From their choices, a sparse embedding model (SPoSE; Zheng et al. 2019, Hebart et al. 2020) learns 40 sparse dimensions.

biorxiv.orgEccentricity, but not category, organizes infant ventral temporal cortexScene- and word-selective regions in adult ventral temporal cortex occupy consistent locations across individuals and align with eccentricity biases for peripheral versus central parts of the visual f...

On held-out judgments, we find the learned dimensions explain twice the variance of the prelabeled annotations (scene, bodies, actions, social-affective): r² = .25 vs 40 learned dimensions: r² = .51

Kathy Garcia reposted

sarah tung

@sarahtung.bsky.social

1/ New preprint! 🧠👶 "Eccentricity, not category, organizes infant ventral temporal cortex." With Emily M Chen, Cleo Tay, Kylie Syzmanski, and @camerontellis.bsky.social www.biorxiv.org/content/10.6...

Kathy Garcia reposted

Rhodri Cusack

@rhodricusack.bsky.social

10 Sept

We're hiring! 2 postdocs + 3 PhD students to join InfantNeuroAI at Trinity College Dublin: awake infant fMRI & OPM-MEG, plus computational models of how infants learn. Aims: understand infant visual cognition, infant neuroimaging methods, more energy-efficient learning 🧵 www.cusacklab.org/vacancies

Kathy Garcia reposted

Jane Li 🦖

@janeli.bsky.social

17 Sept

🦀New preprint! (w/ @najoung.bsky.social)🦞 Is grammaticality a major organizing principle of NLM representations? We show that many NLMs exhibit abstract rep. separation for grammaticality. We believe this work addresses debates about confounds in measuring model gram. knowledge. [1/10]

Kathy Garcia reposted

3 Aug

Looking forward to #CCN2026 - check out the presentations from our lab!

Kathy Garcia reposted

werkenbij.uva.nlVacancy — PhD Position in NeuroAI for Video Perception in the Human Brain<p><span>Are you interested in using AI to unravel the mysteries of the brain? Do you want to perform cutting-edge NeuroAI research and leverage deep learning to understand human vision? Then check out the vacancy below and apply for a PhD position in this exciting research direction.</span></p>

14 May

Heading to #vss2026? Check out the presentations from our lab @vssmtg.bsky.social

Kathy Garcia reposted

Iris Groen

@irisgroen.bsky.social

16 Jan

I have a PhD opening for my #VIDI BrainShorts project 📽️🧠🤖! Are you or do you know an ambitious, recent (or almost) MSc graduate with a background in NeuroAI and interest in large-scale data collection and video perception? Check out our vacancy! (deadline Feb 15). werkenbij.uva.nl/en/vacancies...

3 Dec 2025

Excited to share that I’m at #NeurIPS2025 this year! 🧠 I’ll be presenting my work at UniReps on Social Tokens where we explore how to inject socially relevant visual features into language models to improve social reasoning and alignment. If you’re around, come say hi Feel free to DM for a chat! ☕️

Kathy Garcia reposted

JHU Cognitive Science

@jhucogsci.bsky.social

30 Oct 2025

The department of Cognitive Science @jhu.edu is seeking motivated students interested in joining our interdisciplinary PhD program! Applications due 1 Dec Our PhD students also run an application mentoring program for prospective students. Mentoring requests due November 15. tinyurl.com/2nrn4jf9

Kathy Garcia reposted

Doby Rahnev

@dobyrahnev.bsky.social

23 Sept 2025

Our work showing human-like individual differences in perceptual decisions emerge from random weight initializations in deep neural networks has been accepted in two NeurIPS workshops! 🎉 Awesome job by my student @herrickfung.bsky.social in collaboration with the amazing @apurvaratan.bsky.social.

Kathy Garcia reposted

Hannah Small

@hsmall.bsky.social

24 Sept 2025

Excited to share new work with @hleemasson.bsky.social , Ericka Wodka, Stewart Mostofsky and @lisik.bsky.social! We investigated how simultaneous vision and language signals are combined in the brain using naturalistic+controlled fMRI. Read the paper here: osf.io/b5p4n 1/n

Kathy Garcia reposted

Check out this new preprint led by @gkathy.bsky.social showing how a new fine-tuning scheme using human similarity judgements can improve existing video models, making them more human-aligned on different social tasks! arxiv.org/abs/2510.01502

Kathy Garcia@gkathy.bsky.social

🚨New preprint w/ @lisik.bsky.social! Aligning Video Models with Human Social Judgments via Behavior-Guided Fine-Tuning We introduce a ~49k triplet social video dataset, uncover a modality gap (language > video), and close via novel behavior-guided fine-tuning. 🔗 arxiv.org/abs/2510.01502

In follow-up experiments we show this model generalizes better to novel social tasks, and avoids catastrophic forgetting by preserving baseline on action recognition tasks.

github.comGitHub - garciakathy/similarity-judgments-finetuningContribute to garciakathy/similarity-judgments-finetuning development by creating an account on GitHub.

Together this work shows how different types of human similarity judgments can be leveraged to improve video models. We also share our large-scale video similarity judgment dataset, and code for hybrid triplet/RSA behavior guided fine-tuning: github.com/garciakathy/...

Hybrid fine-tuning substantially increases match to human judgment on held-out videos and surpasses the best language model baseline. The hybrid loss > triplet-only loss and > RSA-only loss.

After fine-tuning, the video model explains both captures more shared variance with language models AND captures more unique variance in human judgments, indicating it learned both language-like semantics and additional visual social nuances.

Despite the task being purely visual, caption embeddings from a language model predict human similarity better than any pretrained video model (e.g., mpnet-base-v2 > TimeSformer-base).

We fine-tune a video transformer with a novel hybrid objective function = Triplet loss (local constraints) + RSA Loss (global Pearson-correlation over pairwise distances), which captures both local and global similarity structure. We use Low Rank Adaptation to reduce the number of trainable params.

Kathy Garcia reposted

Christina Sartzetaki

@sargechris.bsky.social

Excited to be presenting this paper at #ICLR2025 this week! Come to the poster if you want to know more about how human brains and DNNs process video 🧠🤖 📆 Sat 26 Apr, 10:00-12:30 - Poster session 5 (#64) 📄 openreview.net/pdf?id=LM4PY... 🌐 sergeantchris.github.io/hundred_mode...

📢 Excited to announce our paper at #ICLR2025: “Modeling dynamic social vision highlights gaps between deep learning and humans”! w/ @emaliemcmahon.bsky.social, Colin Conwell, Mick Bonner, @lisik.bsky.social ‬ ‪📆 Thur, Apr, 24: 3:00-5:30 - Poster session 2 (#64) ‬ ‪📄 bit.ly/4jISKES%E2%8... [1/6]

📹 While most model features (like architecture or training objective) did not affect performance, we saw a big advantage for video versus image models along the lateral stream. But no model tested could predict anterior lateral stream responses well. [5/6]

🚀 Together, this work highlights a major gap in AI's ability to match human social vision, and underscores the importance of developing AI models in dynamic social contexts [6/6]

🧠 We benchmarked 350+ image, video, and language models against human behavioral and neural responses to dynamic, social videos. [3/6]