Post

A new, highly recommended article from @rasbt.bsky.social: "Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention" -> magazine.sebastianraschka.com/p/recent-dev... #MLsky

Thanks!!

No problem, I am very grateful for your articles (and videos etc), I learn so much from them. BTW: A few days ago I stumbled across this "Simple Attention Networks" concept from Cactus, mostly for one-shot tool use, solely based on the provided context. -> github.com/cactus-compu...

Dunno if that fits in your model garden, but I found this drastic and targeted approach kind of refreshing.