Post

Gemma 4 quantization-aware training (QAT) models are now available, bringing AI performance directly to edge devices and consumer GPUs. These checkpoints are optimized with quantization-aware training to dramatically reduce memory requirements and unlock high-speed local inference. 🧵

Instead of traditional post-training quantization, these models employ custom loss functions and targeted fine-tuning to minimize precision error. This approach delivers a massive reduction in disk and memory footprint while fully preserving the exceptional quality of Gemma 4.

For mobile and edge hardware, a novel quantization schema maximizes efficiency through channel-wise quantization, targeted 2-bit decoding layers, and static activations. The text-only Gemma 4 E2B model requires less than 1 GB of memory.

Ecosystem integrations are live today across popular developer tools including Hugging Face, Llama.cpp, Ollama, MLX, LM Studio, NVIDIA, vLLM, Unsloth, and LiteRT-LM.