Post

A new paper dropped from DeepMind: Deliberation in Latent Space via Differentiable Cache Augmentation The trouble is, it's not very readable. I tried making a thread here, but it got far too long, so it's a blog now: timkellogg.me/blog/2024/12...

It does seem like latent space is the most efficient place to do deliberation - like deep consideration before putting your thoughts into words.

it does. but every time i start thinking in that direction, i can’t convince myself i’m not engaging in magic thinking. like maybe i’m just hoping the algorithms will save me without really understanding the problem. and i’m not sure **how** to understand latent space at scale