Post

Gonna try local LLM on my old hardware - 5950x, 1080Ti (12GB), 32GB RAM. I wonder if there's anything that could replace any "haiku" usage for claude, with at least 10t/s.

15/10/26, 6:13 am0 reposts2 likes

Michal

@arathunku.com

I'm getting 13t/s with www.liquid.ai/blog/lfm2-5-... !!!

liquid.aiLFM2.5-24B-A2B: Built for Fast On-Device Agents | BlogLFM2.5-24B-A2B: an open-weight 24B reasoning model with 2B active parameters, built for fast, private on-device agents. Beats Qwen3.6-35B-A3B on LiveClaw.

Michal

@arathunku.com

doesn't look worthy even for AI judge for bash calls...

Pawel Jozefiak

@joozio.bsky.social

Depends which haiku jobs. Mine only kept the bounded ones local: in a warm-cache test Qwen3.5 4B classified 8 of 10 prompts correctly at 0.89 seconds average. Classification and triage is where I'd start on the 1080Ti too, then see what else holds up.

Waterreptile19

@waterreptile19.bsky.social

With 11GB on the 1080 Ti, a 7 to 8B model at Q4 or Q5 should comfortably clear 10 tok/s. Small Qwen or Gemma variants are good picks for quick Haiku-style tasks.

Michal

@arathunku.com

are they haiku quality though?