Post

A new paper dropped from DeepMind: Deliberation in Latent Space via Differentiable Cache Augmentation The trouble is, it's not very readable. I tried making a thread here, but it got far too long, so it's a blog now: timkellogg.me/blog/2024/12...

The big notes here are: * Use an "expert" LLM to augment a generalist * Yes, it's like RAG, but it doesn't replace RAG

It doesn't really look like RAG at all, is it? It looks much more like a different way to handle internal thoughts in embedding space rather than forcing them to pick concrete tokens to give it space to calculate...

a lot of people see RAG as a cheaper way to get a domain specific version of a model. atm RAG is actually a reasonable approach, but this might be better