Post

correct i’ve been saying this for a couple months. RL is driving towards specialization my hunch is it’s temporary and something will shift again back towards generalization, but for now.. buckle up!

This is really interesting, a middle way between RAG, Lora's and fine tuning.

the neat part with Cartridges is you can do it with any off-the-shelf model with standard inference tooling (uh, needs to be open weights) it’s just stacking the KV cache, that’s already a supported feature in vLLM & sglang and you can still prefill on top, so it’s purely context extension

they stuffed 484K tokens into 128K context

Cartridges are cool, but you need a good dataset right? Like, your training dataset needs to cover all of your latent space. You can write a prompt that is general for all animals, and then collapse it into a cartridge training on most of them, but if you left out emus would emu queries still work?