Arvind Nagaraj

For years, I died a little inside every time I taught the Transformer model, grudgingly accepting that the elegant loop of the RNN was dead.

The Hierarchical Reasoning Model (HRM) isn't just another model. It's a deep synthesis. It marries the iterative soul of an RNN (minus the BPTT nightmare) with the raw power of modern Attention. I wrote a deep dive on why this is a full-circle moment for me, going back to the RNN finetuning days.

medium.comThe Loop is Back: Why HRM is the Most Exciting AI Architecture in YearsYears ago, I sat in Jeremy Howard’s FastAI class, right at the dawn of a new era. He was teaching us ULMFiT, a method he (& Sebastian…

It's a story about why QKV is magic, my love for the loop, and why HRM might be the blueprint for the next generation of AI reasoning. My post, written with the help of an LLM (the irony!), is here. I poured my heart into this one: medium.com/@gedanken.th... #AI #DeepLearning #RNN #Transformer #HRM

For years, I died a little inside every time I taught the Transformer model, grudgingly accepting that the elegant loop of the RNN was dead.

But how does it fix mistakes buried deep in the past? By not letting them stay in the past. Each new "Thinking Session" (the M-loop) starts with the flawed result of the last one. It forces the model to confront its own errors until the logic is perfect.

What makes HRM truly special is its ability to "think fast and slow."Its ACT module isn't just a stop signal; it's a cognitive engine that learns to allocate effort. It's the closest we've come yet to embodying Prof. Kahneman's vision of a System 1/2 mind in code.

For years, I died a little inside every time I taught the Transformer model, grudgingly accepting that the elegant loop of the RNN was dead.

The Hierarchical Reasoning Model (HRM) isn't just another model. It's a deep synthesis. It marries the iterative soul of an RNN (minus the BPTT nightmare) with the raw power of modern Attention.

So how does HRM work? Imagine a tiny,2-person company. 🧠 A strategic CEO (H-module) who thinks slow, sees the big picture, and sets the overall strategy. ⚡️ A diligent Worker (L-module) who thinks fast, executing the details of the CEO's plan. This separation allows for truly deep, iterative thought.

For years, I died a little inside every time I taught the Transformer model, grudgingly accepting that the elegant loop of the RNN was dead.

Then, last month, a paper dropped that changes everything. This is the architecture I've been waiting for since 2018. A thread on HRM. 🧵

arxiv.orgrStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep ThinkingWe present rStar-Math to demonstrate that small language models (SLMs) can rival or even surpass the math reasoning capability of OpenAI o1, without distillation from superior models. rStar-Math achie...

9 Jan 2025

🔥🔥 MCTS rollout pruning, python interpreter verifier and iterative self improvement of intermediate steps during each round of training. Brilliant stuff this💪 rStar-Math is the kind of paper I wish to see more of!

mr. TIM@timkellogg.me

rStar-Math takes qwen2.5 7b & 1.5b as well as qhi3 3.8b and fine tunes them for math they’re able to exceed o1-preview on math benchmarks (with the 7B) the magic sauce seems to be in co-evolving the SLM (the main LLM) and the PPM (Process Preference Model, the verifier) arxiv.org/abs/2501.04519

Arvind Nagaraj reposted

Patricia Murrieta-Flores

@patymurrieta.bsky.social

17 Dec 2024

(1/7) For a while we've been working on an ambitious problem: The National Archive of Mexico #AGN holds 58 linear km of documents. Only a drop of this ‘ocean’ has been studied due to many challenges. But great news: we are now unlocking this information! A thread 🧵 (1/8) #HTR #AI #CulturalHeritage

Arvind Nagaraj reposted

Serge Belongie

@serge.belongie.com

19 Dec 2024

Computer Vision: Fact & Fiction is now available on YouTube 🙌🏼 I made a playlist for it with the seven chapters. Enjoy this time capsule from two decades ago!

youtube.comComputer Vision: Fact & Fiction - YouTubeTitle Computer Vision: Fact and Fiction Year of Publication 2005 Author(s) Marília Maschion Vincent Rabaud Serge Belongie URL https://vision.ucsd.edu/publica...

Serge Belongie@serge.belongie.com

In 2005 two UCSD students (Marília Maschion and Vincent Rabaud) and I made a documentary about the state-of-the-art in Computer Vision, based on interviews with researchers at CVPR in San Diego, CA (1/5)

19 Dec 2024

I like how the new gemini 2.0 thinking model insists like a child...lol

Arvind Nagaraj@narvind.bsky.social

Easy for humans. Hard for LLMs. Even harder sans tool use and iteration👇 Use any mathematical signs wherever you need: 2 + 2 + 2 = 6 3 3 3 = 6 4 4 4 = 6 5 5 5 = 6 6 6 6 = 6 7 7 7 = 6 8 8 8 = 6 9 9 9 = 6

3 Dec 2024

RoPE has been the one 💯 genuine upgrade to the vanilla Vaswani transformer. This beautiful blogpost by Chris Fleetwood explains the significance and how rotations of Q & K preserves meaning(magnitude) while encodes relative positions(angle shift) 🔥🔥

4. RoPE takes this operation from the beginning of the input to inside the attention operation itself. 5. There are 2 benefits: the semantic meaning of the token is not corrupted. We only rotate the vector, preserving the magnitude.

fleetwood.devfleetwood.devfleetwood.dev

3 Dec 2024

6. V is not rotated. Only Q and K are rotated relative to each other. Farther tokens now have a larger angle between them. 7. The encoding signal is not going to die out. It can be preserved by doing it as part of the softmax dot product attn. 8. What a gorgeous 😍 idea...

Link: fleetwood.dev/posts/you-co...

3 Dec 2024

TL;DR: 1. We need a way to encode token positions when feeding them as input into the transformer 2. We could just concat 1,2,3 etc. but this doesn't scale for variable lengths 3. Noam Shazeer showed show sin and cos waves can produce a beautiful pattern that encodes relative positions bw tokens.

1 Dec 2024

Why does ChatGPT refuse to say "David Mayer" ?? 🤔 I have tried a bunch of ways and it refuses to!! 😭

Arvind Nagaraj reposted

Karsten Roth

@confusezius.bsky.social

AI Conference DL Countdown

🤔 Can you turn your vision-language model from a great zero-shot model into a great-at-any-shot generalist? Turns out you can, and here is how: arxiv.org/abs/2411.15099 Really excited to this work on multimodal pretraining for my first bluesky entry! 🧵 A short and hopefully informative thread:

Arvind Nagaraj reposted

@dlcountdown.bsky.social

29 Nov 2024

SIGGRAPH'25 (form): 48 days. RSS'25 (abs): 49 days. SIGGRAPH'25 (paper-md5): 55 days. RSS'25 (paper): 56 days. ICML'25: 62 days. RLC'25 (abs): 77 days. RLC'25 (paper): 84 days. ICCV'25: 97 days.

29 Nov 2024

We should give this place a serious try... It may work 🙏

29 Nov 2024

So many good X tweets on sparse autoencoders that are not here on 🦋 Trying to find those people and follow them here.

Look there is chaos everywhere. More bitterness. "Us versus Them" factions finger pointing, name calling and blame shifting more than ever before. Still, I am thankful for all those I get to interact with, learn from and share my ideas and happiness with. I'm thankful for you all🙏

Arvind Nagaraj reposted

Andreas Kirsch

@blackhc.bsky.social

huggingface.coalpindale/two-million-bluesky-posts · Datasets at Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.

Has anyone shared huggingface.co/datasets/alp... as a torrent yet? Happy to support that effort

👌👌

Jeremy Howard @howard.fm

Oops I missed one letter in my code. Corrected version: ``` from atproto import * def f(m): print(m.header, parse_subscribe_repos_message(m)) FirehoseSubscribeReposClient().start(f) ```

I'm going to the profiles of each person who threatened the HF guy to see what earth shattering ideas they came up with. So far it's mostly junk and therefore the dataset wouldn't have been of much value anyway. (Unless you're training 'task vectors' for junk, to improve your models.)

ificl.github.ioMultiFoleyVideo-Guided Foley Sound Generation with Multimodal Controls

Mikolov... Vinyals... For the next 8-9 years it's going to be fairly easy to predict the Neurips test of time awards. Kaiming He 2025 anyone??

Arvind Nagaraj reposted

Andrew Owens

@andrewowens.bsky.social

27 Nov 2024

We generate a soundtrack for a silent video, given a text prompt! For example, we can make a cat's meow sound like a lion's roar or a typewriter sound like a piano. Paper: arxiv.org/abs/2411.17698 Webpage: ificl.github.io/MultiFoley/ Led by @czyang.bsky.social! bsky.app/profile/czya...

Ziyang Chen@czyang.bsky.social

🎥 Introducing MultiFoley, a video-aware audio generation method with multimodal controls! 🔊 We can ⌨️Make a typewriter sound like a piano 🎹 🐱Make a cat meow like a lion roars! 🦁 ⏱️Perfectly time existing SFX 💥 to a video. arXiv: arxiv.org/abs/2411.17698 website: ificl.github.io/MultiFoley/

Arvind Nagaraj reposted

Smerity

@smerity.bsky.social

Replying to @giffmana.ai

Caddy's a great choice. I love that it stretches from local poking and prodding to full production. The Caddy local file server (i.e. `caddy file-server --listen 0.0.0.0:8000 --browse`) overtook `python -m http<dot>server` for me given it looks nicer and has grid view for images :)

o1 style reasoning can solve most of these. But 8 is tricky. Deepseek's model (inner monologue thinking tokens) are super interesting to watch. But the CoT trajectories take it to 2 incorrect solutions before it runs out thinking time: It either adds an extra 8 or uses cube roots. Can't nest like👇

Arvind Nagaraj@narvind.bsky.social

This thing's "mind" looks like a ferret on crack 🤣 I had to explain it to the poor thing!

mark-riedl.medium.comTransformers: OriginsAn unofficial origin story of the transformer neural network architecture.

🔥🔥🔥 blogpost from @markriedl.bsky.social articulating the history of modern NLP leading up to the LM task, embedding semantics into vectors and learning the underlying representations via gradient descent, the Vaswani transformer and all the way to RLHF alignment and inference tricks like CoT.

Mark Riedl@markriedl.bsky.social

I have converted a portion of my NLP Online Masters course to blog form. This is the progression I present that takes one from recurrent neural network to seq2seq with attention to transformer. mark-riedl.medium.com/transformers...

Multimodality is super 🔥 now! The sense of smell got added to the mix as we see companies like Osmo build a digital nose and a smell printer. IMHO, the digital nose (input) will get adopted sooner than the smell printer(output) like the other senses - vision, touch. Perhaps audio is an exception 🤔

blog.dottxt.coCoalescence: making LLM inference 5x faster

@dottxtai.bsky.social is doing 👌 work with structured data. Their new blog post describes Coalescence - using Finite state machines to generate JSON output upto 5X faster during LLM inference!🔥 Pydantic model -> JSON schema -> regex -> FSM -> Selective sampling! 👏👏 blog.dottxt.co/coalescence....

arxiv.orgOn Subjective Uncertainty Quantification and Calibration in Natural Language GenerationApplications of large language models often involve the generation of free-form responses, in which case uncertainty quantification becomes challenging. This is due to the need to identify task-specif...

Thanks Jeremy, for sharing the instructions! I have moved to a custom domain as well 😊

Jeremy Howard @howard.fm

I've moved to a custom domain, so I'm now @howard.fm. All links to my previous bsky id (jph) should have auto-updated. I created a new "jph" id to avoid anyone taking over external links to my account. Directions here for anyone interested in doing the same: bsky.social/about/blog/4...

I'm going to read this paper next. But my starting point is that ok scepticism. I feel the uncertainty is beautifully captured by the final softmax's token logprobs. Let me see what the authors actually have to say...

Shubhendu Trivedi@shubhendu.bsky.social

This is a simple and good paper, which somehow nobody working on these things cites, or even seems to be aware of arxiv.org/abs/2406.05213 It is simple idea that seems useful; it formulates the subjective uncertainty for natural language generation in a decision-theoretic setup.

🤯🤯 This is amazing!! By now most people understand and appreciate what a major update RoPE was, to the QK operation in the vannila transformer, that struggled to retain the signal from sin-cos positional embeddings. Finding the Bfloat16 bug and fixing it in this manner is genius work! 🔥🔥🔥

Sung Kim@sungkim.bsky.social

They propose a fix called AnchorAttention: A plug-and-play method that improves long-context performance, reduces training time by over 50%, and preserves the model's general capabilities.

huggingface.cohuggingface.co

Truth has been spoken

Christoph Molnar@christophmolnar.bsky.social

No one can explain stochastic gradient descent better than this panda.

Arvind Nagaraj reposted

merve

@merve.bsky.social

23 Nov 2024

icymi hugging face transformers has classes for agentic workflows 🤗 huggingface.co/docs/transfo... $ pip install transformers[agents] read agentic applied ML recipes in our cookbook and start building! huggingface.co/learn/cookbo...

Clem Delangue 🤗@clem.hf.co

Should HF do more agent stuff? If so, what would be useful?

22 Nov 2024

I'm trying to understand what's going on with Alibaba's new Marco-o1 model. At first glance it looked like they managed to introduce valuable "reasoning" tokens, compute a reward score (softmax of argmax token logprob over top 5 potential candidate tokens) And finally added a reflection phrase

I don't think I'll need to calculate the confidence logprob scores etc - that should have been part of training. I may be missing something basic - please help me out.

23 Nov 2024

Looks like it takes a special system prompt to trigger that behavior! huggingface.co/AIDC-AI/Marc...

My understanding of the o1 model is that the "reasoning chains of tokens" are learned via RL during model training. And any straight decoding leads to similar reasoning chains during inference, which OpenAI "hides" from the user and just displays the "Thinking 🤔" message to the user.

22 Nov 2024

But the Marco o1 doesn't seem to produce any long chains of reasoning. Just regular answers. My gut feel is that inference time reasoning tokens should be helpful. I believe Noam's claims @polynoamial.bsky.social in this regard. I'd like to play with models that exhibit this behavior.

So far, so good. It appears that the combination of full fine-tuning with their datasets + reasoning tokens + reflection tokens during training should result in comprehensive, step-by-step reasoning answers during inference. But that's not the case with their HF model.

22 Nov 2024

The model spits out straight answers. No Reasoning steps of any sort. The git repo code is just straight HF model+tokenizer load as usual and autoregressive decoding.

The phrase "Wait! Maybe I made some mistakes! I need to rethink from scratch." allows for expansion of the "thoughts tree". It looks like this self-critic step(inspired from the popular ASU papers) adds additional search paths for MCTS to explore.

Arvind Nagaraj reposted

Maria Antoniak

@mariaa.bsky.social

20 Nov 2024

Today I learned that ⌥ and click in your terminal moves your cursor 🤯