GerstnerLab

biorxiv.orgInterpretable compositional computation with recurrent neural networksFlexible cognition utilizes reusable components to enable rapid adaptation of behavior to different contexts or tasks. Analysis of artificial neural networks trained on multiple tasks suggested that t...

15 Aug

Excited to share that I'll be starting as a junior professor and Emmy Noether awardee at the University of Göttingen this October! 🥳 🚀 I'll be hiring #phd students and #postdoc across #machinelearning, #neuroscience and #cogsci: modirlab.github.io/open-positio... Please spread the word! 🙏

GerstnerLab reposted

Shuqi Wang

@shuqiw.bsky.social

15 July

Our paper "Rarely categorical, highly separable representations along the cortical hierarchy" is now out in Nature! www.nature.com/articles/s41... In this paper, we studied whether the brain is well-organized and interpretable, both globally and locally. (See thread below.)

GerstnerLab reposted

Both brains and RNNs can re-use components of computation across similar tasks or contexts. But what exactly are those “shared components”? How can they be used to solve several tasks? We address these questions in a new preprint with @avm.bsky.social! Link: www.biorxiv.org/content/10.6...

How can the brain learn the hidden hierarchical structure from high dimensional data? In our latest work, we use synthetic datasets to analyze two classes of bio-plausible learning rules: variants of Direct Feedback Alignment, and local self-supervised learning. We find only the latter succeeds.

Local self-supervised rules (CLAPP, LPL, layerwise SimCLR) succeed with the same data efficiency as supervised backprop. These algorithms need no backward pass because they optimize separate objectives at each layer. The weight-updates can be written as Hebbian-like plasticity rules.

arxiv.orgSelf-supervised local learning rules learn the hidden hierarchical structure of high-dimensional dataThe brain learns abstract representations of high-dimensional sensory input, but the plasticity rules that enable such learning are unknown. We study biologically plausible algorithms on the Random Hi...

Paper is available on arxiv.org/abs/2605.18557 This work is done by: Ariane Delrocq, Zihan Wu @zihan-wu.bsky.social, Guillaume Bellec @bellecguill.bsky.social and Wulfram Gerstner. We thank all members of LCN and PCSL at EPFL for their insightful discussion

We use the Random Hierarchy Model (RHM, arxiv.org/abs/2307.02129) to generate a family of artificial datasets with an intrinsic hierarchical structure and adjustable degree of complexity. It can be seen as a controlled proxy for vision or language data.

Direct Feedback Alignment and its variants fail to learn hidden hierarchies, even with near-optimal feedback weights. The reason: DFA ignores input-specific masking - the derivative of forward activations. In controlled experiments, we demonstrate its necessity in approximating backpropagation.

We focus primarily on these two classes because they satisfy the biological constraints: no weight transport or symmetric feedback architecture, no need for slow convergence of activities.

22 June

📂 We're sharing code of a classic from the lab! A biologically inspired RL model from Frémaux, Sprekeler @sprekeler.bsky.social & Gerstner (2013, PLOS CB): a continuous-time actor–critic using spiking neurons that learn to navigate a water maze

github.comGitHub - realwsq/Fremaux2013: This project implements a biologically inspired continuous-time actor–critic model using spiking neurons. The agent learns the water-maze task. Reference paper: Frémaux, ...This project implements a biologically inspired continuous-time actor–critic model using spiking neurons. The agent learns the water-maze task. Reference paper: Frémaux, Sprekeler, Gerstner (2013, ...

22 June

✦ Online weight updates via the TD-LTP rule ✦ Critic & actor use SRM0 neurons ✦ Place cells encode position as Poisson spikes Code available here 👇 github.com/realwsq/Fremaux2013 Implemented by @shuqiw.bsky.social #Neuroscience #MachineLearning #OpenScience

GerstnerLab reposted

Guillaume Bellec

@bellecguill.bsky.social

17 June

Accepted at ICML 2026. I am extremely proud of this work. Local learning is starting to work really well! The theoretical perspective of local learning minimizing an emergent global objective has been eye opening Congrats @zihan-wu.bsky.social and Ariane Delrocq! Great team

Zihan Wu@zihan-wu.bsky.social

Can we match self-supervised backpropagation using local learning rules? We show it is possible in our new paper accepted by ICML. We achieve: 1. theoretical equivalence to BP in a controlled setup 2. new SOTA for local learning across image datasets 3. same performance as BP on multiple datasets

16 June

How far are biologically-plausible local learning rules from backpropagation (BP)? In our new ICML paper we look at local self-supervised learning and find how to better align its gradients to BP. We achieve competitive performance on various datasets!

Zihan Wu@zihan-wu.bsky.social

@sobeckerneuro.bsky.social

10 June

Ever heard of the lottery ticket hypothesis? Our new paper shows that lottery tickets are not a useful metaphor to explain the success of overparameterized neural networks - and suggests an alternative metaphor: escape dimensions

Flavio Martinelli@flavioh.bsky.social

NEW PAPER. Why do larger networks train better? "Because they contain more candidate *sub*networks that can learn the task" → lottery tickets This popular explanation uses an appealing but misleading metaphor🧵 We propose an intuitive alternative grounded in theory: escape dimensions

GerstnerLab reposted

Sophia Becker

27 May

Incredibly grateful to @cmiehl.bsky.social and @gjorjulijana.bsky.social for their thoughtful and spot-on Preview "Novelty beyond counting" of our recent work in Neuron with @modirshanechi.bsky.social and @gerstnerlab.bsky.social! 🧠✨ It's a real honor 🙏 www.cell.com/neuron/fullt...

GerstnerLab reposted

cell.comFinding clues to circuit structure in population dynamics and single-neuron selectivityIn this issue of Neuron, Pezon et al. introduce neural circuit models with flexible connectivity structure that can generate low-dimensional population dynamics with different distributions of single-...

8 May

Unbelievably honoured to read Tatiana Engel's (@engeltatiana.bsky.social) wonderfully written Preview on our work "Linking neural manifolds to circtuit structure in recurrent networks" (with @lpezon.bsky.social & @gerstnerlab.bsky.social) in this issue of Neuron www.cell.com/neuron/fullt... 🙏

GerstnerLab reposted

Gaute Einevoll

@gauteeinevoll.bsky.social

28 Mar

Episode #39 in #TheoreticalNeurosciencePodcast: On modeling neural population activity with mean-field models – with Tilo Schwalger theoreticalneuroscience.no/thn39 How can mean‑field models be systematically derived from the underlying microscopic dynamics of individual neurons?

GerstnerLab reposted

EPFL Brain Mind Institute

@epfl-brainmind.bsky.social

1 Apr

Novelty is not just about whats new, but also what feels new given past experience. New study from Sophia Becker in Wulfram Gerstner’s @epfl-brainmind.bsky.social lab posits a model showing how similarity between familiar and novel stimuli shapes exploration and learning - doi.org/10.1016/j.ne...

GerstnerLab reposted

Louis Pezon

@lpezon.bsky.social

13 Mar

Presenting a poster tomorrow at Cosyne 26: [3-033] Compositional computation via shared latent dynamics in low-rank RNNs. With @avm.bsky.social, we explore how RNNs can re-use the same dynamics across different tasks, and what it implies for their connectivity and neural activity.

GerstnerLab reposted

Louis Pezon

@lpezon.bsky.social

6 Mar

Excited to share our new paper to be published in Neuron! With Valentin Schmutz @bio-emergent.bsky.social and Wulfram Gerstner @gerstnerlab.bsky.social, we explore how circuit structure in RNNs shapes network computation and single-neuron responses. www.sciencedirect.com/science/arti...

sciencedirect.comLinking neural manifolds to circuit structure in recurrent networksDimensionality reduction methods are widely used in neuroscience to investigate two complementary aspects of neural activity: the distribution of sing…

GerstnerLab reposted

Flavio Martinelli

@flavioh.bsky.social

4 Dec 2025

🧵Excited to present our latest work at #Neurips25! Together with @avm.bsky.social, we discover 𝐜𝐡𝐚𝐧𝐧𝐞𝐥𝐬 𝐭𝐨 𝐢𝐧𝐟𝐢𝐧𝐢𝐭𝐲: regions in neural networks loss landscapes where parameters diverge to infinity (in regression settings!) We find that MLPs in these channels can take derivatives and compute GLUs 🤯

GerstnerLab reposted

Alexander van Meegen

@avm.bsky.social

5 Nov 2025

This was a lot of fun! From my side, it started with a technical Q: what's the relation between two-side cavity and path integrals? Turns out it's a fluctuation correction - and amazingly, this also enable the "O(N) rank" theory by @david-g-clark.bsky.social and @omarschall.bsky.social. 🤯

David G. Clark@david-g-clark.bsky.social

Now in PRX: Theory linking connectivity structure to collective activity in nonlinear RNNs! For neuro fans: conn. structure can be invisible in single neurons but shape pop. activity For low-rank RNN fans: a theory of rank=O(N) For physics fans: fluctuations around DMFT saddle⇒dimension of activity

journals.aps.orgConnectivity Structure and Dynamics of Nonlinear Recurrent Neural NetworksThe structure of brain connectivity predicts collective neural activity, with a small number of connectivity features determining activity dimensionality, linking circuit architecture to network-level...

Lab members are at the Bernstein conference @bernsteinneuro.bsky.social with 9 posters! Here’s the list: TUESDAY 16:30 – 18:00 P1 62 “Measuring and controlling solution degeneracy across task-trained recurrent neural networks” by @flavioh.bsky.social

WEDNESDAY 14:00 – 15:30 P4 25 “Rarely categorical, always high-dimensional: how the neural code changes along the cortical hierarchy” by @shuqiw.bsky.social P4 35 “Biologically plausible contrastive learning rules with top-down feedback for deep networks” by @zihan-wu.bsky.social

P4 52 “Coding Schemes in Non-Lazy Artificial Neural Networks” by @avm.bsky.social

P2 65 “Rate-like dynamics of spiking neural networks” by Kasper Smeets

WEDNESDAY 12:30 – 14:00 P3 4 “Toy Models of Identifiability for Neuroscience” by @flavioh.bsky.social P3 55 “How many neurons is “infinitely many”? A dynamical systems perspective on the mean-field limit of structured recurrent neural networks” by Louis Pezon

TUESDAY 18:00 – 19:30 P2 2 “Biologically informed cortical models predict optogenetic perturbations” by @bellecguill.bsky.social P2 12 “High-precision detection of monosynaptic connections from extra-cellular recordings” by @shuqiw.bsky.social

GerstnerLab reposted

28 Sept 2025

New in @pnas.org: doi.org/10.1073/pnas... We study how humans explore a 61-state environment with a stochastic region that mimics a “noisy-TV.” Results: Participants keep exploring the stochastic part even when it’s unhelpful, and novelty-seeking best explains this behavior. #cogsci #neuroskyence

GerstnerLab reposted

19 Sept 2025

🎉 "High-dimensional neuronal activity from low-dimensional latent dynamics: a solvable model" will be presented as an oral at #NeurIPS2025 🎉 Feeling very grateful that reviewers and chairs appreciated concise mathematical explanations, in this age of big models. www.biorxiv.org/content/10.1... 1/2

sciencedirect.comContext selectivity with dynamic availability enables lifelong continual learning“You never forget how to ride a bike”, – but how is that possible? The brain is able to learn complex skills, stop the practice for years, learn other…

🧠 “You never forget how to ride a bike”, but how is that possible? Our study proposes a bio-plausible meta-plasticity rule that shapes synapses over time, enabling selective recall based on context

In experiments (models & simulations), we showed how this approach supports stable retention of old tasks while learning new ones (split CIfar-100, ASC…)

Work led by Martin Barry with the supervision of Wulfram Gerstner and Guillaume Bellec @bellecguill.bsky.social

We designed a Gating/Availabilty model that detects selective neurons - most useful neuron for the task - during learning, shunt activity of the others (Gating) and decrease the learning rate of task selective neuron (Availability)

We designed a Bio-inspired Context-specific gating of plasticity and neuronal activity allowing for a drastic reduction in catastrophic forgetting. We also show the capacity of our model of both forward and backward transfer! All of this thanks to the shared neuronal activity across tasks.

GerstnerLab reposted

dlvr.itMerits of Curiosity: A Simulation StudyAbstract‘Why are we curious?’ has been among the central puzzles of neuroscience and psychology in the past decades. A popular hypothesis is that curiosity is driven by intrinsically generated reward signals, which have evolved to support survival in complex environments. To formalize and test this hypothesis, we need to understand the enigmatic relationship between (i) intrinsic rewards (as drives of curiosity), (ii) optimality conditions (as objectives of curiosity), and (iii) environment structures. Here, we demystify this relationship through a systematic simulation study. First, we propose an algorithm to generate environments that capture key abstract features of different real-world situations. Then, we simulate artificial agents that explore these environments by seeking one of six representative intrinsic rewards: novelty, surprise, information gain, empowerment, maximum occupancy principle, and successor-predecessor intrinsic exploration. We evaluate the exploration performance of these simulated agents regarding three potential objectives of curiosity: state discovery, model accuracy, and uniform state visitation. Our results show that the comparative performance of each intrinsic reward is highly dependent on the environmental features and the curiosity objective; this indicates that ‘optimality’ in top-down theories of curiosity needs a precise formulation of assumptions. Nevertheless, we found that agents seeking a combination of novelty and information gain always achieve a close-to-optimal performance on objectives of curiosity as well as in collecting extrinsic rewards. This suggests that novelty and information gain are two principal axes of curiosity-driven behavior. These results pave the way for the further development of computational models of curiosity and the design of theory-informed experimental paradigms.

25 Aug 2025

So happy to see this work out! 🥳 Huge thanks to our two amazing reviewers who pushed us to make the paper much stronger. A truly joyful collaboration with @lucasgruaz.bsky.social, @sobeckerneuro.bsky.social, and Johanni Brea! 🥰 Tweeprint on an earlier version: bsky.app/profile/modi... 🧠🧪👩‍🔬

Open Mind@openmindjournal.bsky.social

GerstnerLab reposted

13 Aug 2025

Attending #CCN2025? Come by our poster in the afternoon (4th floor, Poster 72) to talk about the sense of control, empowerment, and agency. 🧠🤖 We propose a unifying formulation of the sense of control and use it to empirically characterize the human subjective sense of control. 🧑‍🔬🧪🔬

youtu.beFrom Spikes To RatesYouTube video by Gerstner Lab

8 Aug 2025

Is it possible to go from spikes to rates without averaging? We show how to exactly map recurrent spiking networks into recurrent rate networks, with the same number of neurons. No temporal or spatial averaging needed! Presented at Gatsby Neural Dynamics Workshop, London.

journals.aps.orgEmergent Rate-Based Dynamics in Duplicate-Free Populations of Spiking NeuronsCan spiking neural networks (SNNs) approximate the dynamics of recurrent neural networks? Arguments in classical mean-field theory based on laws of large numbers provide a positive answer when each ne...

8 Aug 2025

Work lead by Valentin Schmutz (@bio-emergent.bsky.social), in collaboration with Johanni Brea and Wulfram Gerstner.

GerstnerLab reposted

Lucas Gruaz

@lucasgruaz.bsky.social

11 June 2025

Excited to present at the PIMBAA workshop at #RLDM2025 tomorrow! We study curiosity using intrinsically motivated RL agents and developed an algorithm to generate diverse, targeted environments for comparing curiosity drives. Preprint (accepted but not yet published): osf.io/preprints/ps...

osf.ioOSF

GerstnerLab reposted

Sophia Becker

@sobeckerneuro.bsky.social

11 June 2025

Stoked to be at RLDM! Curious how novelty and exploration are impacted by generalization across similar stimuli? Then don't miss my flash talk in the PIMBAA workshop (tmr at 10:30, E McNabb Theatre) or stop by my poster tmr (#74)! Looking forward to chat 🤩 www.biorxiv.org/content/10.1...

biorxiv.orgRepresentational similarity modulates neural and behavioral signatures of noveltyNovelty signals in the brain modulate learning and drive exploratory behaviors in humans and animals. While the perceived novelty of a stimulus is known to depend on previous experience, the effect of...

GerstnerLab reposted

biorxiv.orgHigh-dimensional neuronal activity from low-dimensional latent dynamics: a solvable modelComputation in recurrent networks of neurons has been hypothesized to occur at the level of low-dimensional latent dynamics, both in artificial systems and in the brain. This hypothesis seems at odds ...

9 June 2025

Our new preprint 👀

Matteo Carandini@carandinilab.net

The firing of neural populations is high-dim even if their subthreshold activity is low-dim! This work by @bio-emergent.bsky.social and @haydari.bsky.social shows how, with a solvable model, a data analysis technique, and data from mouse visual cortex: www.biorxiv.org/content/10.1...

GerstnerLab reposted

Alexander van Meegen

@avm.bsky.social

10 June 2025

Interested in high-dim chaotic networks? Ever wondered about the structure of their state space? @jakobstubenrauch.bsky.social has answers - from a separation of fixed points and dynamics onto distinct shells to a shared lower-dim manifold and linear prediction of dynamics.

Jakob Stubenrauch@jakobstubenrauch.bsky.social

(1/3) How to analyse a dynamical system? Find its fixed points, study their properties! How to analyse a *high-dimensional* dynamical system? Find its fixed points, study their properties! We do that for a chaotic neural network! finally published: doi.org/10.1103/Phys...

doi.orgFixed point geometry in chaotic neural networksUnderstanding the high-dimensional chaotic dynamics occurring in complex biological systems such as recurrent neural networks or ecosystems remains a conceptual challenge. For low-dimensional dynamics...

GerstnerLab reposted

Gaute Einevoll

@gauteeinevoll.bsky.social

7 Dec 2024

Episode #22 in #TheoreticalNeurosciencePodcast: On 50 years with the Hopfield network model - with Wulfram Gerstner theoreticalneuroscience.no/thn22 John Hopfield received the 2024 Physics Nobel prize for his model published in 1982. What is the model all about? @icepfl.bsky.social

GerstnerLab reposted

actu.epfl.chBrain models draw closer to real-life neuronsResearchers at EPFL have shown how rough, biological spiking neural networks can mimic the behavior of brain models called recurrent neural networks. The findings challenge traditional assumptions and...

22 Jan 2025

A cool EPFL News article was written about our recent neurotheory paper on spikes vs rates! Super engaging text by science communicater Nik Papageorgiou. actu.epfl.ch/news/brain-m... Definitely more accessible than the original physics-style, 4.5-page letter 🤓 journals.aps.org/prl/abstract...

GerstnerLab reposted

actu.epfl.chLearning from the unexpectedA researcher at EPFL working at the crossroads of neuroscience and computational science has developed an algorithm that can predict how surprise and novelty affect behavior.

10 Jan 2025

Super excited to see my PhD thesis featured by EPFL! 🎓 actu.epfl.ch/news/learnin... P.S.: There's even a French version of the article! It feels so fancy! 😎 👨‍🎨 🇫🇷 actu.epfl.ch/news/apprend...

GerstnerLab reposted