Post

You can easily save up to 65% of compute while improving performance on reasoning tasks 🤯 👀 Meet EAGer: We show that monitoring token-level uncertainty lets LLMs allocate compute dynamically - spending MORE on hard problems, LESS on easy ones. 🧵👇

EAGer works by monitoring token entropy during generation. High entropy token → It branches to explore new paths (reusing prefixes). Token with low entropy → It continues a single path. We cap at M sequences/prompt, saving budget on easy ones without regen. Training-free! 🧵3/

The fun part: EAGer-adapt reallocates saved budget to "saturating" prompts hitting the M cap, no labels needed! – Training & Verification-Free 🚀 Full EAGer uses labels to catch failing prompts, lowering threshold to branch or add sequences. Great for verifiable pipelines! 🧵4/

Results: Across 3B-20B models, EAGer cuts budget by up to 80%, boosts perf 13% w/o labels & 37% w/ labels on AIME. As M scales, EAGer consistently: 🚀 Achieves HIGHER Pass@k, ✂️ Uses FEWER tokens than baseline, 🕺 Shifts the Pareto frontier favorably across all tasks. 🧵5/