Robert Rosenbaum

apply.interfolio.com Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

We're hiring for multiple tenure track or tenured positions in the Department of Applied and Computational Mathematics and Statistics at U Notre Dame. Apply soon because we'll start reviewing applications in a couple of weeks. Please feel free to share this post. apply.interfolio.com/187311

apply.interfolio.com Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

7 Oct 2025

We're hiring 5 T/TT faculty in Neuroscience, including Computational Neuroscience, at U Notre Dame We'll start reviewing applications very soon, so if you're thinking about applying, please apply now/soon! apply.interfolio.com/173031

apply.interfolio.com Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

30 Sept 2025

I'm posting this again for anyone who might have missed it last time: Notre Dame is hiring 5 tenure or tenure-track professors in Neuroscience, including Computational Neuroscience, across 4 departments. Feel free to reach out with any questions. And please share! apply.interfolio.com/173031

apply.interfolio.com Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

3 Sept 2025

The University of Notre Dame is hiring 5 tenure or tenure-track professors in Neuroscience, including Computational Neuroscience, across 4 departments. Come join me at ND! Feel free to reach out with any questions. And please share! apply.interfolio.com/173031

arxiv.orgOja's plasticity rule overcomes several challenges of training neural networks under biological constraintsDeep neural networks have achieved impressive performance through carefully engineered training strategies. Nonetheless, such methods lack parallels in biological neural circuits, relying heavily on n...

New preprint with my postdoc, Navid Shervani-Tabar, and former postdoc, Marzieh Alireza Mirhoseini. Oja’s plasticity rule overcomes challenges of training neural networks under biological constraints. arxiv.org/abs/2408.08408

Finally, we meta-learned pure plasticity rules with no weight transport, extending our previous work. When Oja's rule was included, the meta-learned rule _outperformed_ pure backprop.

In other words: Plasticity rules like Oja's let us go beyond studying how synaptic plasticity in the brain can _match_ the performance of backprop. Now, we can study how synaptic plasticity can _beat_ backprop in challenging, but realistic learning scenarios.

Vanilla RNNs trained with pure BPTT fail on simple memory tasks. Adding Oja's rule to BPTT drastically improves performance.

We find that Oja's rule works, in part, by preserving information about inputs in hidden layers. This is related to its known properties in forming orthogonal representations. Check the paper for more details.

We propose that plasticity rules like Oja's rule might be part of the answer. Adding Oja's rule to backprop improves learning in deep networks in an online setting (batch size 1).

We often forget how important careful weight initialization is for training neural nets because our software initializes them for us. Adding Oja's rule to backprop also eliminates the need for careful weight initialization.

In our new preprint, we dug deeper into this observation. Our motivation is that modern machine learning depends on lots of engineering hacks beyond pure backprop: gradients averaged over batches, batchnorm, momentum, etc. These hacks don't have clear, direct biological analogues.

For example, a 10-layer ffwd network trained on MNIST using online learning (batch size 1) performs poorly when trained with pure backprop. How does the brain learn effectively without all of these engineering hacks?

A lot of work in "NeuroAI," including our own, seeks to understand how synaptic plasticity rules can match the performance of backprop in training neural nets.

In previous work on this question, we meta-learned linear combos of plasticity rules. In doing so, we noticed something intersting: One plasticity rule improved learning, but its weight updates weren't aligned with backprop's. It was doing something different. That rule is Oja's plasticity rule.

Robert Rosenbaum reposted

Nosrat Mohammadi

@nosrat.bsky.social

16 May 2025

I made this figure panel size guide to avoid thinking about dimensions every time. Apparently this post is going to be a 🧵! So feel free to bookmark it and save some time of yours.

1 May 2025

Well deserved. Congratulations, Adrienne!

UW Neurobiology & Biophysics@uw-nbio.bsky.social

Congratulations to Adrienne Fairhall, Prof. of Neurobiology and Biophysics, Adjunct Prof. of Applied Mathematics, and Adjunct Prof. of Physics, who has just been elected to the National Academy of Sciences! 🍾 🎉

21 Apr 2025

High-Dimensional Dynamics in Low-Dimensional Networks. New preprint with a former undergrad, Yue Wan. I'm not totally sure how to talk about these results. They're counterintuitive on the surface, seem somewhat obvious in hindsight, but then there's more to them when you dig deeper.

21 Apr 2025

Recent work by Vincent Thibeault, @allard.bsky.social, and Patrick Desrosiers shows that many networks arising in nature have an (approximate) low dimensional structure in the sense that the singular values of their adjacency matrices are dominated by a small number of large values.

mitpress.mit.eduModeling Neural Circuits Made Simple with PythonAn accessible undergraduate textbook in computational neuroscience that provides an introduction to the mathematical and computational modeling of neurons an...

Somewhat old news here, but since I'm switching over from the other site: Happy to announce that my Computational Neuroscience textbook was published by MIT Press. The text and code is freely accessible: mitpress.mit.edu/978026254808... (click Open Access) drive.google.com/drive/folder... (1/n)

Additional topics and models are covered in a second appendix.

amazon.comModeling Neural Circuits Made Simple with Python (Computational Neuroscience Series)Amazon.com: Modeling Neural Circuits Made Simple with Python (Computational Neuroscience Series): 9780262548083: Rosenbaum, Robert: Books

An open access version is available on the publishers site (see above). Code to produce all figures is available on Colab and github. Hard copies of the book are available from the publisher or from Amazon. www.amazon.com/Modeling-Neu...

It's great for the classroom, but also for readers from other fields (such as ML/AI) who want to learn about biological neural networks. The content is based on a course I've continued to develop for 10 years. (2/n)

The main text covers a minimal thread of concepts needed to build up from ion channels to neural networks. The book assumes no background in biology, only a basic background in math (e.g., calculus and matrices). All other mathematical background is covered in an appendix. (3/n)