Post

I spent 2 months learning about quantization and am extremely proud of the post I've written about it. I think these are some of the nicest visuals I've ever made, and I love how this compression technique invented in 1898 is being used on the bleeding edge in 2026. ngrok.com/blog/quantiz...

Lil' gallery of some of the visuals to entice you to click and justify my continued employment. The post starts by introducing what models actually _are_ under the hood, and what leads them to become so large they can't fit in your laptop's RAM.

Then I talk about how computers represent floating point numbers, the trade-offs involved, and how LLMs kinda don't care that much about precision (but not in that way.)

As part of the work on this post, I opened up 6 popular open models and analysed their parameters. This is a histogram that shows that model parameter values cluster close to 0, with the vast majority of values being very small.

This diagram seems a bit slow on my system (moving the slider is rather laggy). Not a deal breaker at all, the article is a delight, but maybe something you might consider (my PC is rather beefy, or well, it was 3 years ago I guess :D) btw I'm running Firefox.

Could you try now? I think I improved it.

Smooth as butter. Just out of curiosity: what was it?

I also had a lot of fun learning the different ways that LLM degradation is measured after they've been compressed to a quarter or even an eighth of their original size. One of these measures, KL divergence, is all about how well 2 probability distributions overlap.

If you read it, I'd love your feedback! I'm always trying to improve along all vectors: writing, visuals, accessibility, depth, story arc. ❤️

Beautiful work.

It's great! I was planning to start writing a series on quantization for my blog but this is so much better 😅 Now I don't need to write anything, I can just link to this.