Post

mr. TIM@timkellogg.me

someone got Kimi K3 running on a consumer laptop at a blazing 0.32-0.34 tok/sec github.com/sqliteai/waste

github.comGitHub - sqliteai/waste: Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine. - sqliteai/waste