Post

@barredspirals.comint.su have you tried Strata? it's not my computer, and I can't test it thoroughly rn, but strata bench reports ~2k t/s prefill ~80 t/s decode on 12GB VRAM + 64GB RAM for qwen3.8-flash-next iq3_s, and testing with some leetcode prompts via curl seems to confirm it

I haven't tried it yet since there's no Vulkan support and i haven't been willing to go fight rocm again. if it's *that* fast it might actually be worth it tho (seems insane; those are the same specs I have and I get like 6tps decode, tops)

yeah I have absolutely no slightest idea how it managed that. I have only tried the bench and curling stuff like "Implement an AVL tree in C..." though

like, I couldn't get these speeds from qwen3.6-35b-a3b on the same machine with llama.

hm i don't see support for my Radeon 6700 XT mentioned anywhere