Post

Are you using open models on a regular basis? If yes, I’d love to hear about it! - what’s your use-case? - local or through an inference provider? - how has the experience evolved over time? - anything you miss from the closed models, if you’ve used them? Would appreciate reposts for visibility 🙏

I run a portion of all of my automated work through open models via an inference provider. When the observed difference in output becomes negligible, I demote that task to an open model full-time. Partly cost optimization, but mostly to ensure my tooling is still valuable when token subsidies end.

It's kind of hard to describe without providing a ton of background, but basically all of the code generation steps in our automated issue workflow are now done by cheaper models. The issue shaping, implementation validation, etc are still done by high-end models.