Unsloth AI

Introducing Unsloth Desktop 🦥 The first desktop app to run and train models locally. • Open-source. Runs on Mac, Windows, Linux • Supports MLX, diffusion image/video, audio, GGUF • Connect Claude Code + Codex to local LLMs • 50% more accurate self-healing tool calls Download on unsloth.ai + GitHub

MiniMax-M3.1-Flash can now run locally! We made 2-bit GGUF retain 91.5% top 1 accuracy while being 45% smaller. Run on 150GB RAM. MiniMax-M3.1-Flash is the strongest open model for its size & among the best overall. GGUF: huggingface.co/unsloth/Mini... Guide: unsloth.ai/docs/models/...

NVIDIA releases Nemotron-3.5-Bolt-E4B, a new model that runs locally on 6GB RAM. 💚 Bolt-E4B can run on edge devices like phones and delivers top-tier performance for its size. Trained on Gemma 4, it supports vision. GGUF: huggingface.co/unsloth/NVID... Guide: unsloth.ai/docs/models/...

Run Flint locally on 24GB RAM via Unsloth Dynamic GGUFs! Flint is the strongest MoE model under 50B parameters. Recursive’s Apache-2.0 model has 39.8B parameters (2.5B active). Run at 125 t/s on Mac, 150 on RTX Spark GGUF: huggingface.co/unsloth/flin... Guide: unsloth.ai/docs/models/...

DeepSeek V4.1-Flash-1017 can now run locally! 🐳 We made 1-bit quants retain ~88.2% accuracy while shrinking size from 508GB to 250GB (−52%). Run on a 128GB+ RAM device. V4.1 Flash 1017 is the strongest open model to date. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Deep...

Meta releases Muse Spark Open, a new SOTA open model. Muse-Spark-Open-816B-A42B supports text, images, video and audio input with 1M context. Run via Dynamic 2-bit Unsloth GGUFs on 256GB RAM. GGUF: huggingface.co/unsloth/Muse... Guide: unsloth.ai/docs/models/...

NVIDIA releases 2 new models: Nemotron-3.5-Bolt and Nemotron-3.5-Super-VL. Nemotron-3.5-Bolt-74B-A9B runs on 24GB RAM. It delivers top-tier performance for its size. Super-VL-120B-A12B runs on 82GB RAM. Bolt GGUF: huggingface.co/unsloth/NVID... Guide: unsloth.ai/docs/models/...

You can now train your own Decision model with our free notebook! 💡 Qwen3.5-4B will generate decisions instead of text on just 8GB VRAM locally. Learn to data prep (state, questions, gold answers), train, serve. Notebook: colab.research.google.com/github/unslo... Guide: unsloth.ai/docs/basics/...

Windows now has sandboxing! Microsoft released an open-source repo, mxc, for sandboxed code execution. We collaborated with Windows to add mxc OS level sandboxing to Unsloth which adds just <100 ms of overhead. GitHub: github.com/unslothai/un... Guide: unsloth.ai/docs/new/stu...

You can now train your own Decision model like Jev locally! We increased Qwen3.5 0.8B’s aggregate accuracy from 20.7% to 74.3% across 3 decision benchmarks - on just 4GB VRAM. Turn any LLM, Qwen3.8, Gemma 4 into decision models. GitHub: github.com/unslothai/un... Guide: unsloth.ai/docs/basics/...

Google releases EmbeddingGemma 2, a new open model that runs locally on 0.5GB RAM. The 740M parameter embedding model combines a 270M text model with vision (170M) + audio (300M). Run & train the model via Unsloth. GGUF: huggingface.co/unsloth/embe... Guide: unsloth.ai/docs/models/...

You can now run Laya Decision models locally on just 4GB RAM! 🔥 Works on CPU, Mac, Windows, Linux and GPU setups. Serve Laya through a Jev-compatible API via Unsloth Desktop. GitHub: github.com/unslothai/un... Guide: unsloth.ai/docs/models/...

Unsloth has surpassed 500M model downloads on Hugging Face! 🦥🤗 Qwen3.8-27B GGUF is already Unsloth’s #1 most-downloaded model ever. Thanks for all your support!

Qwen-Image-2.1 can now run locally on 12GB VRAM with Unsloth GGUFs! The 7B parameter model performs on par with Nano Banana 2.0. Run our Dynamic FP8 or GGUFs for higher quality via diffusers, Unsloth Desktop & more. GGUF: huggingface.co/unsloth/Qwen... Guide: unsloth.ai/docs/models/...

You can now train and run 500+ models locally with our Unsloth Docker image! 🐳 Use our new GUI or notebooks workflow. No setup required. Works on NVIDIA and AMD. Guide: unsloth.ai/docs/get-sta... GitHub: github.com/unslothai/un...

Qwen3.8-27B Unsloth GGUF is now the #1 most-liked GGUF of all time! The model hit 10M downloads and 3.7K likes in just 24 days on Hugging Face - all thanks to you. 🤗🦥 GGUF: huggingface.co/unsloth/Qwen... Guide: unsloth.ai/docs/models/...

We made GLM-5.3-Flash run 3.3x faster locally! Local GGUF inference is now 1.6–3.4× faster with optimized decoding and bonus multi-token prediction. Run 3-bit on 128GB setups via Unsloth Desktop or llama.cpp. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/GLM-...

Qwen3.8-Flash can now run 1.7× faster locally with MTP!⚡️ GGUFs can reach 170 tokens/s on a RTX PRO 6000. MTP enables Qwen3.8-Flash-Next ~1.3–1.7× faster inference with no accuracy change. GGUFs: huggingface.co/unsloth/Qwen... Guide: unsloth.ai/docs/models/...

GLM-5.3 can now be run locally! The 2-bit model retains ~81% accuracy after we shrunk it from 1.51TB to 239GB (-83% size). Run on a 256GB Mac or RAM/VRAM setups. GLM-5.3 is the strongest open model to date. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/GLM-...

GLM-5.3-Flash can now be run locally! ✨ Run 3-bit on 128GB RAM via Unsloth GGUF. GLM-5.3-Flash (ox-alpha) rivals Claude Opus 4.8 on DeepSWE, coding & agentic benchmarks. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/GLM-...

Qwen3.8-Flash can now be run locally! 🔥 The 125B MoE model outperforms Claude-Opus-4.6 (Max). Run on 75GB RAM via Unsloth GGUFs. Qwen3.8-Flash-Next enables CPU RAM / unified mem setups to deliver near VRAM speeds. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Qwen...

GLM-5.3-Flash, also known as ox-alpha, is out now! Zai's GLM-5.3-Flash is a new 320B parameter multimodal open model with 18B active parameters. GLM-5.3-Flash approaches Claude 4.8 Opus on coding and agentic benchmarks. GGUF coming soon!

You can now fine-tune Qwen3.8-27B for free with our notebook! 🔥 Local training works on 24GB VRAM. Unsloth trains Qwen3.8-27B 1.5x faster with 50% less VRAM. GitHub: github.com/unslothai/un... Qwen3.8-27B Notebooks + Guide: unsloth.ai/docs/models/...

Qwen announces Qwen3.8-Flash-Next, a new open-weight multimodal MoE model. 💜 The model will be released tomorrow and we are working on Unsloth day zero support. Qwen3.8-Flash-Next is built on the new Qwen4 architecture.

We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy. Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks. We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM. Blog: unsloth.ai/docs/basics/... GGUF: huggingface.co/unsloth/Qwen...

Qwen3.8-27B Unsloth GGUF is now the #2 trending model on Hugging Face with 2.7M downloads! 💗 Unsloth also reached #3 trending on GitHub! Thanks so much for the love! Model: huggingface.co/unsloth/Qwen... GitHub: github.com/unslothai/un...

Qwen3.8-27B can now be run locally! ✨ Run on 17GB RAM via Unsloth Dynamic GGUFs. Qwen3.8-27B is by far the strongest model for its size. We also uploaded NVFP4 quants. GGUF: huggingface.co/unsloth/Qwen... Guide: unsloth.ai/docs/models/...

You can now fine-tune Meta Muse Glimmer 30B for free! 🔥 Our free notebook also supports GRPO RL training. Unsloth trains Muse Glimmer 1.5× faster with 50% less VRAM vs FA2 setups. Train locally with 24GB VRAM. Guide: unsloth.ai/docs/models/... Notebooks: unsloth.ai/docs/models/...

Qwen3.8 can now be run locally! 🔥 We shrank Qwen3.8-2.4T-A95B from 4.9TB to 397GB (-91% size) via Dynamic 1-bit by selectively quantizing layers. Run on 410GB+ RAM/VRAM via Unsloth Desktop. Qwen3.8 rivals GPT-5.6 Sol. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Qwen...

2-bit Muse Glimmer GGUF made 100+ tool calls on just 14GB RAM. 🔥 Muse Glimmer did a complete repo bug hunt for 5 mins nonstop with: evidence, repro, fix, tests and a PR writeup. Run and train it in Unsloth. GitHub repo: github.com/unslothai/un...

Meta releases Muse Glimmer, a new 30B open model that runs on 18GB RAM. Apache 2.0 licensed and built for agentic coding, it supports vision and is the strongest agentic model for its size. Run and train the model via Unsloth. GGUF: huggingface.co/unsloth/Muse... Guide: unsloth.ai/docs/models/...

DeepSeek-V4-Flash can now run 2× faster locally with DSpark! ⚡️ DSpark enables V4-Flash-0731 GGUFs to generate ~1.4–2× faster with no accuracy change. DeepSeek-V4-Flash-0731 can reach at 120 tokens/s. GGUFs: huggingface.co/unsloth/Deep... Guide: unsloth.ai/docs/models/...

Qwen just announced Qwen3.8-27B along with Qwen3.8-Max! 🔥 Qwen3.8-27B will run locally on 17GB RAM/VRAM setups and is expected to be the best performing model for its size.

You can now run Inkling-Small, a new 276B model by Thinking Machines. Inkling-Small is the strongest open model for its size and runs local on 128GB RAM. Apache-2.0 Licensed, it has image, audio + 1M context support. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Inkl...

We compared 1-bit Kimi K3 to Claude Opus 5 and GPT 5.6. We gave 4 models the same prompt: Create a glass aquarium whose side panel develops a visible crack and then bursts. 1-bit Kimi K3 GGUF ran locally on 4x B200s at 36 tok/s. GitHub repo: github.com/unslothai/un...

Kimi K3 can now be run locally! ✨ The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size). Run on a Mac Studio + 128GB RAM device. Kimi K3 is the strongest open model to date. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Kimi...

We signed the Open Weights letter because we believe the future of AI should be shaped by everyone, not controlled by a select few. That belief has always been at the heart of Unsloth: everyone should be able to train and run models on their own local device.

Introducing Unsloth for AMD 🚀 You can now train & run LLMs on your AMD hardware • We collaborated with AMD to enable you to train & run 500+ models on AMD GPUs • Works on Windows, WSL, Linux • Train Qwen, Gemma on just 3GB VRAM GitHub: github.com/unslothai/un... Blog: unsloth.ai/docs/basics/...

Gemma 4 is now faster and much more accurate! 🚀 Google made huge improvements to tool-calling and chat accuracy, reliability + speed. To get fixes, re-download our updated GGUF, MLX, NVFP4 quants! Unsloth quants: huggingface.co/collections/... Gemma 4 Guide: unsloth.ai/docs/models/...

Inkling, a new 975B parameter open model is here! From Thinking Machines, Inkling supports image, audio, text & 1M context. We quantized Inkling to Dynamic 1-bit (-86% size) and retained 74.2% of top-1% accuracy. Run on 280GB. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/inkl...

We’re releasing Gemma 4 NVFP4 quants that run 1.5× faster on your GPU. Gemma-4-12B NVFP4 works on 11GB VRAM. 26B-A4B hits 13K tok/s (B200). Unsloth NVFP4 enables faster, more accurate 4-bit Blackwell inference. Blog: unsloth.ai/docs/basics/... Gemma NVFP4: huggingface.co/collections/...

We collaborated with AWS on a complete guide to LLM Quantization and Deployment. Learn about: • Model formats, dynamic quants & making your own • Choosing GGUF, NVFP4 or FP8 • Picking the right tools & deploy on AWS SageMaker • Benchmark quality, latency & cost Read: aws.amazon.com/blogs/machin...

We’re releasing new Qwen3.6 quants that run 2.5× faster on your GPU. ⚡ Qwen3.6-27B NVFP4 runs on 24GB VRAM. 35B-A3B can hit 17,561 tok/s (B200). We also improved accuracy, tool calling, agent use, and looping. Guide: unsloth.ai/docs/models/... Qwen3.6 NVFP4: huggingface.co/collections/...

DeepSeek-V4 can now run locally with Unsloth GGUFs! 🐳 Run lossless DeepSeek-V4-Flash on 168GB RAM. 3-bit works on 110GB Mac, RAM, VRAM setups. Run via Unsloth Studio or llama.cpp. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Deep...

What’s your go-to local model right now?

1-bit GLM-5.2 GGUF vs. Claude 4.8 Opus vs. GPT-5.5 We gave 3 models the same prompt and compared one-shot outputs. The 1-bit GLM-5.2 GGUF ran locally on a Mac Studio M3 Ultra 256GB RAM at ~21.6 tok/s. Which do you like best? GGUF: huggingface.co/unsloth/GLM-... Guide: unsloth.ai/docs/models/...

GLM-5.2 can now be run locally! 🔥 The 2-bit model retains ~82% accuracy after we shrunk it from 1.51TB to 238GB (-84% size). Run on a 256GB Mac or RAM/VRAM setups. GLM-5.2 is the strongest open model to date. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/GLM-...

You can now run Kimi K2.7 Code locally! 🌘 We shrank the 1T model to 325GB (-48%) via Dynamic 2-bit where important layers are upcasted. Run at >40 tok/s on 330GB RAM/VRAM setups. Run full precision on 610 GB. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Kimi...