Post

got bonsai and 27b running decently on my Rx 6700 (10GB VRAM), packages as a nix flake so anyone can run it with minimal work 9b -- 813 tokens/s prefill, 84 tokens//s 27B -- 267 tokens/s prefill, 26 tokens/s github.com/alyraffauf/b...

github.comGitHub - alyraffauf/bonsai-buddy: Run Prism Bonsai 2 9B and 27B locally with Nix, tuned AMD GPU profiles, and OpenCode.Run Prism Bonsai 2 9B and 27B locally with Nix, tuned AMD GPU profiles, and OpenCode. - alyraffauf/bonsai-buddy

elynxabeth holmes ⎔

@freyja-lynx.dev

... me with my 6700 XT...

aly

@aly.codes

you could def use this and run 27b with a longer context window.

elynxabeth holmes ⎔

@freyja-lynx.dev

gonna play with this right meow

aly

@aly.codes

fun fact I pretend to have your GPU so rocm runs bc its not supported lol

24/10/26, 6:33 pm0 reposts3 likes

No replies yet.