Post

On one hand, you can do what Jev does now with any traditional LLM (even as small as qwen 9b) - ask me anything. But to get it done at the quality and speed of Jev is the big use-case. AI developers in the meantime were excited about AST in a harness so industry dev is like a year behind anyway.

Side note: we were doing a lot of this with functiongemma last year and you can tune that openweights models for things like intents in an app, but it's roughly the same running cost with lower quality than LoRA on a small model.

Is it easy/possible to guarantee a structured response like that with traditional LLMs? I've never tried or needed to but it always felt like it would be hard.

when you can dramatically reduce both cost and latency of results from models, you can unlock a _lot_ of user-facing outcomes that just wouldn't be possible with current LLMs without ruining user experience

like we are talking about a model that returns in milliseconds results that often beat frontier models with _zero-shot_ prompting. claiming this is "just hype" is imo extremely close-minded

I think it’s more like “this implementation is nothing special” Seeing that this was a Big Opportunity should probably still get you something tho Maybe Jev the model is not technically impressive or unique and is supplanted entirely in the next few months

i'm kinda torn on it honestly on one hand it's a very cool generic model on the other hand a lot of classification tasks are very specific and there are better specific models for it still cool though as it encourages experimentation

that is true, but there's a lot of cost that comes with both training and running classification models, especially at scale. but its also not just for classification tbqh!

the concern I have is people using jev as a hammer and assuming every classification problem is a nail for it. if there is existing training data, jev will at least cost more and/or perform worse. but in all scenarios where no training data exists, which is most of them, jev is very valuable.

new tech will take time for people to understand. it interfaces closer with data science which has a very long history of modeling, so most people who just got into this stuff from LLMs are very confidently asserting things that jev can do but really shouldn’t. but jev has a lot of uses still!

i do suspect it will get over used, but i’m also very confident it will help a ton with the training of new classification models. having something to use as a ground zero for training new models is pretty great tbh

its product thinking vs engineering thinking technically, Jev is *not* interesting, and that’s why all the labs overlooked it but also, it changes everything

I feel like Jev is neat to people who learned what LLMs were before they learned what classifiers were.

you’re not blown away by a classifier that requires zero labeling or training?? (and is almost as cheap)