Post

amd halo AI/strix halo 128GB inference box with customized Qwen3.8 Flash-Next Strata settings, built on NixOS+K3s. Basically ChatGPT in a flake. github.com/devon-system...

github.comGitHub - devon-systems/kanto: Strix Halo ML inference + infrastructure IaC for NixOS+K3sStrix Halo ML inference + infrastructure IaC for NixOS+K3s - devon-systems/kanto

25/10/26, 9:41 pm3 reposts35 likes

aly

@aly.codes

Designed this way bc I intend to add another one to the cluster :3

jae (hallowed)

@fubarchitect.com

hell yeah

Frank Blank

@frank8465.bsky.social

I needed this. Badly. Holy shit.

Obi Ryder

@obiryder.bsky.social

Putting the whole setup in a flake makes this interesting beyond a one-off inference box. When you add the second machine, are you aiming to serve separate models on each node or spread one model across both? Curious what you want K3s to handle first.