Will Redman

New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.6... 1/

Collectively, we view our results as supporting the results from @azvollan.bsky.social and @masahironakano.bsky.social + @caswell.bsky.social + @clopathlab.bsky.social that "predictive looking" representations (e.g., left-right alternating theta sweeps in MEC) need not be predictive in origin 11/

We're excited about these results and what they might mean for how we understand predictive coding in MEC! We're also very open to any feedback either here or wredman4@jh.edu 12/

New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.6... 1/

Our RNN results suggest that predictive grid cells might be involved in computations more general than planning, e.g. maintaining the local grid cell circuit. To test this, we looked for signatures of predictive grid coding in the MEC data collected while the rats were asleep 9/

While noisy, we do find one recording where the cells identified as predictive grid cells during open field exploration show a bias towards encoding prospective locations on the population level toroidal manifold during sleep 10/

New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.6... 1/

And in RNN's trained to path-integrate, we also find predictive grid coding, despite not having a loss function that explicitly enforces prediction/planning 7/

We validated that the units in the RNN really were encoding prospective information by measuring their dependence on movement direction, while controlling for the same allocentric location. We find predictive grid units are significantly more sensitive to future paths than standard grid units 8/

New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.6... 1/

With two very motivated and hard working students (Aarav Sinha + Jommy Fasehun) we analyzed MEC recordings and RNNs trained to path-integrate, looking for predictive grid coding 5/

In @rjgardner.bsky.social + @m-bmoser.bsky.social + @edvardmoser.bsky.social 's publicly available MEC data (science.org/doi/10.1126/...) we find robust predictive grid cell coding 6/

New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.6... 1/

I was really fascinated/surprised by the discovery of Ouchi & Fujisawa (2024) on the existence of neurons in MEC that encode future locations with grid cell-like ratemaps. This seemed consistent with MEC playing a role in predictive coding/goal-directed navigation 3/

But it was unclear: 1) how widely present these predictive grid cells are in rodent MEC, 2) how predictive grid cells should be understood, given that standard grid cells have already been connected with predictive coding 4/

New preprint out (first from my new group!) 🚨 "Predictive Grid Cell Coding Does Not Require Explicit Planning" biorxiv.org/content/10.6... 1/

science.orgPredictive grid coding in the medial entorhinal cortexThe entorhinal cortex represents allocentric spatial geometry and egocentric speed and heading information required for spatial navigation. However, it remains unclear whether it contributes to the pr...

TLDR: Predictive grid cells (science.org/doi/10.1126/...): 1) are a robust feature of rodent MEC 2) emerge in RNNs trained to path-integrate (w/o any explicit prediction loss) 3) may be present even during sleep, suggesting they are playing a more general role beyond prediction 2/

Will Redman reposted

Hanson Lu

@hansonlu.bsky.social

Our new method, Hessian Null Space Continuation (HNC)! Around any trained network there is a large connected set of weights that compute nearly the same function. HNC walks through this set along flat directions of the Hessian. 1/3

Ann Huang@annhuang42.bsky.social

How much representational diversity exists around a trained network? A lot, it turns out! Moving a ViT’s weights by 1.3% while preserving its predictions, its representations end up less similar to the original than a random-init ViT’s ‼️ New paper: Hessian Null Space Continuation (HNC) 💫🧵 (1/12)

Will Redman reposted

Mitchell Ostrow

@neurostrow.bsky.social

Super excited about this new work from @annhuang42.bsky.social! We developed a method for identifying different solutions to a task in neural networks of all types by walking along the flat directions of the loss landscape. Doing so revealed a ton of hidden solutions nearby in weight space!

Ann Huang@annhuang42.bsky.social

Really exciting work from @annhuang42.bsky.social that was a lot of fun to be a part of! Recent work has revived the interest in finding “diverse solutions” - i.e. DNNs that perform similarly on the same task but do so with very different internal representations 1/3

Ann Huang@annhuang42.bsky.social

We asked, “how hard is it to find a new solution?” Turns out, not very hard! By moving along the null space of the Hessian, we efficiently explored the solution space of problems ranging from memory tasks (RNNs) to image classification (ViTs) to goal-directed navigation (RL) 2/3

A lot of really exciting doors open with this method! And I think creative uses of the HNC will be especially revealing on aspects of deep learning

21 Sept

∂DSA finds lots of RNN solutions to the same integration task: Line attractors ✅ Feedforward chains ✅ Oscillations ✅ Unstable dynamics ✅

Mitchell Ostrow@neurostrow.bsky.social

We also built a fast, differentiable form of DSA — ∂DSA. With this, we showed we can train RNNs online with DSA to learn novel, never-before-hypothesized solutions to computational tasks! (Stay tuned for future work on this too!) (8/)