Post

Tim Duffy

@timfduffy.com

Due to sparsity, Kimi K3 has fewer active parameters (104 billion) than GPT-3 (175 billion, same as total parameters)

2/8/26, 2:41 am1 repost42 likes

Antarctus

@antarctus.bsky.social

2 Aug

I find that lightweight local models (like Gemma) are sometimes even better than the first GPT-4. Capacity density is really the metric that has skyrocketed.

Sam Harsimony

@harsimony.bsky.social

2 Aug

And each parameter in Kimi is 4-8x smaller than GPT-3 (idk if GPT-3 was float16 or 32)