Post

Smaine Kahlouch (Smana)
@smana.dev

Open-weight LLMs are catching up to frontier models—fast. On SWE-bench, top open models now sit within ~7 points of Claude Opus 4.7. So I built a self-hosted platform on K8s to see what it takes to run this for real today. 🧵

1 repost5 likes

Nicolas Fonrose
@nfonrose.bsky.social

> The open-weight models that could truly compete (DS V4 Pro, Kimi K2.6) require hardware that's simply out of reach for personal use It could be interesting to extend your platform to allow usage of external resources (from vast.ai for ex) to make these models accessible (until you have the HW)

vast.aiRent GPUs | Vast.aiRent high-performance cloud GPUs at low cost with Vast.ai. Instantly deploy GPU rentals for AI, machine learning, deep learning, and rendering. Flexible pricing, fast setup, and global availability
Smaine Kahlouch (Smana)
@smana.dev

The stack is built for evolution: ✅ vLLM on NVIDIA L4 (fp8) ✅ Crossplane InferenceService (1 YAML = the stack) ✅ KEDA scaling on GPU batch/KV signals ✅ Envoy AI Gateway + Iris for smart semantic routing ✅ Amazon S3 Files for shared POSIX storage