Samuel Lippl

@annaschapiro.bsky.social

3 Aug

Really excited about this project! If you want to chat about repulsion and representational changes in hippocampus, come by our poster at #CCN2026 tomorrow at 9:30!

Anna Schapiro@annaschapiro.bsky.social

Here's where to find the Penn Computational Cognitive Neuroscience Lab at Cognitive Computational Neuroscience next week! See you in NYC!

Samuel Lippl reposted

Anna Schapiro

1 Aug

Here's where to find the Penn Computational Cognitive Neuroscience Lab at Cognitive Computational Neuroscience next week! See you in NYC!

arxiv.orgA mathematical theory of balancing relational generalization and memorizationHumans, animals, and modern machine learning models exhibit impressive abilities to learn complex behaviors and generalize these behaviors to unseen situations. This ability requires us to learn rules...

10 July

Excited to present this paper that was led by Luke today at the CompLearn workshop (2nd poster session, starting at 4). If you want to chat about how simple learning systems can learn relational rules but also memorize exceptions to those rules, come by!

Luke Cheng@lukecheng.bsky.social

🧵 New preprint! "A mathematical theory of balancing relational generalization and memorization" w/ @sflippl.bsky.social Real-world learning requires both learning rules AND memorizing exceptions. We introduce a task + theory for understanding how learning systems do both. arxiv.org/abs/2605.22972

Excited to be presenting this paper tomorrow! If you want to hear more about this, please come to the poster session tomorrow at 2:30pm @icmlconf.bsky.social.

Ida Momennejad@neuroai.bsky.social

Somewhat late but thrilled to share that our paper on Algorithmic Primitives & Compositional Geometry of Reasoning in Language Models was accepted to ICML 2026. Huge thanks to incredible coauthors, esp @sflippl.bsky.social @eberleoliver.bsky.social @thomasmcgee.bsky.social. Camera ready coming soon.

Pretraining + fine-tuning powers modern ML, but we lack a theoretical understanding of how pretraining actually shapes downstream learning. In our new @icmlconf.bsky.social paper, we address this gap! 📅 July 9th, Poster #4502 Session 8! 🧵 arxiv.org/pdf/2602.20062

Main takeaway: Fine-tuning behavior is strongly shaped by the inductive bias inherited from pretraining and network initialization.

This was a great collaboration with Nicolas Anguita, Francesco Locatello, @saxelab.bsky.social, @mmondelli.bsky.social, @flaviamancini.bsky.social, and @clementinedomine.bsky.social. If you want to hear more, please come chat at the poster session tomorrow (starting at 5pm).

→ We derive exact generalization curves via replica theory, linking initialization parameters, data structure, and sample size

Beyond the idealized setting: → We extend the analysis to imperfect pretraining → In ResNets and Transformers, we show that a negative relative scale improves generalization when tasks rely on a sparse subset of pretrained features

→ This induces four distinct fine-tuning regimes, including a previously overlooked regime that enables both reuse and refinement

→ A key control knob: the relative scale of weights across layers

We investigate pretraining + fine-tuning in diagonal linear networks as a tractable model, deriving an exact characterization of fine-tuning behavior.