Post

DeepSeek has put a lot of effort into making their KV cache smaller and smaller, this is important for allowing large efficient batching at long context lengths.

10/9/26, 4:34 pm3 reposts57 likes

1G-N15 ⬡

@1gn15.com

11 Sept

No wonder they have the cheapest cache read costs. And the cheapest cache reach costs usually result in the cheapest overall costs in agentic use.

Tim Duffy

@timfduffy.com

10 Sept

They keep pushing the envelope on sparsity too, really impressive

caleb

@caleb.jasik.xyz

11 Sept

the nemotron model that has a tiny kv cache has been one of the best performing models on my m1 max 64gb mac