Post

Are you using open models on a regular basis? If yes, I’d love to hear about it! - what’s your use-case? - local or through an inference provider? - how has the experience evolved over time? - anything you miss from the closed models, if you’ve used them? Would appreciate reposts for visibility 🙏

1. Extracting summaries/info from paragraph to page-length texts across medium scale (100-1000 books); also framing and answering questions about them. 2. Local. 3. I’ve gotten better at it and have realized that reasoning models can be overkill. 4. I don’t try open for coding and big concept tasks.

Motivation is slightly cost, but more that I can share an open workflow with students and collaborators without making *them* pay or introducing complex accounting.

Not sure I have a well-worked-out theory. I have observed a) non-reasoning models suffice for most summary and extraction tasks b) reasoning models may be slower or more fragile — can overrun token limits and, more tentatively, c) reasoners can sometimes "overthink," overreacting to one detail +

E.g., when I ask gpt-oss:20b to fill in a masked sentence in "an Irish novel," it goes on and on about the Emerald Isle and standing up to the English and &c