Post

Linear attention is fast, but its fixed-size state limits memory This paper expands that matrix into a 3D tensor using two keys, giving an 8x larger recurrent state with only 1.2% more params At 1.3B scale, it improves long-context recall & is 5x faster at 64k context alphaxiv.org/abs/2609.36529