Post

Don't you need a lot of RAM to run it? What if you have 20GB VRAM and 32GB RAM?

When I bought my 3080 I thought I was going overkill. Surely 10gb vram is plenty. I guess I was lucky in one way, equivalent range cards are now easily 3x $

per the model card it's 6B activated out of 125B, plus a 51B n-gram embedding that Qwen calls more amenable to offloading than MoE weights