Post

DeepSeek R1 appears to be a VERY strong model for coding - examples for both C and Python here: simonwillison.net/2025/Jan/27/...

simonwillison.netggml : x2 speed for WASM by optimizing SIMDPR by Xuan-Son Nguyen for `llama.cpp`: > This PR provides a big jump in speed for WASM by leveraging SIMD instructions for `qX_K_q8_K` and `qX_0_q8_0` dot product functions. > > …

Sirius Li

@siriusli.bsky.social

29 Jan 2025

Dumb q: Why are there multiple huggingface options for the same model but with vastly different download sizes? What's the catch with the 16GB option? Thanks in advance!

Simon Willison

@simonwillison.net

That's quantization: it's a trick where you effectively drop the number of figures after the floating point to shrink the overall size of the model, but it comes at a quality cost simonwillison.net/2024/Nov/23/...

simonwillison.netQuantization mattersWhat impact does quantization have on the performance of an LLM? been wondering about this for quite a while, now here are numbers from Paul Gauthier. He ran differently quantized …

29/1/25, 6:31 pm0 reposts1 like

No replies yet.