Post

@barredspirals.comint.su have you tried Strata? it's not my computer, and I can't test it thoroughly rn, but strata bench reports ~2k t/s prefill ~80 t/s decode on 12GB VRAM + 64GB RAM for qwen3.8-flash-next iq3_s, and testing with some leetcode prompts via curl seems to confirm it

27/10/26, 2:48 pm0 reposts2 likes

lexi

@barredspirals.comint.su

I haven't tried it yet since there's no Vulkan support and i haven't been willing to go fight rocm again. if it's *that* fast it might actually be worth it tho (seems insane; those are the same specs I have and I get like 6tps decode, tops)

lexi

@barredspirals.comint.su

like I only get speeds like that from 3–4b models normally

affine

@refinement.systems

yeah I have absolutely no slightest idea how it managed that. I have only tried the bench and curling stuff like "Implement an AVL tree in C..." though

affine

@refinement.systems

like, I couldn't get these speeds from qwen3.6-35b-a3b on the same machine with llama.