Post

If you're unsure what model your machine can handle, you can check axjns.dev llmfit. It's a TUI that profiles your hardware and scores models to tell you exactly what fits in your RAM/VRAM. This is it running on my mac:

iury souza

@iurysouza.dev

what I'm using: bsky.app/profile/iury...

iury souza@iurysouza.dev

Jumped from 20 TPS to 80 TPS ⚡ on my M4 Pro. The trick is that this version uses a MoE architecture with 3 billion active parameters. That makes it way faster. I was running the base model via GGUF (ollama), and now running on MLX (vlm - because this is a vision model too).

20/3/26, 6:27 pm0 reposts0 likes

No replies yet.