Post

Chrome shipped an LLM Prompt API to the web platform. At Mozilla, we oppose this API. Here's why:

you don't need to license Google's model, it's free and it's going to change - every model out there has a very limited life span; any LLM based app needs to be ready for different models the API itself may be half baked and definitely shouldn't be rushed through without consensus either way!

for that matter, their model is already outdated - other vendors already ship (free, open) models that are smaller, faster, and more intelligent the API may be rushed, but the system prompt and model lock-in concerns aren't real - or they are, but you're just describing the reality of using LLMs

Ever wondered why Google is shipping its own 4gb model rather than using the one already on the OS? For… consistency! Which is exactly the problem here.

yes but having different models in different browsers should be a good thing then? so we're encouraged to write prompts and workflows that work with more than one model?

also, using "the one on the OS" would have the same problem, since there is more than one OS and those models will change too, so this problem is not going away, is what I'm saying it's partially why we have open benchmarks for things like tool use - models can try to align on some capabilities

don't get me wrong, I am not defending Google! they clearly shouldn't have rushed this - web standards need time to mature but coming up with acceptable standards required participation by the browser vendors

If Google shipped this in a way that uses the OS-exposed model, so there was immediately no interop on this feature, that would be less problematic than what they actually did.

that's an implementation detail though, it doesn't affect the API other vendors are free to use the OS model, built-in model, local models, a user supplied API key, even providing all of those options it's not like we can standardize the model - this problem exists no matter what Google ships

it occurs to me, this problem is at least partially solvable by making available an open "web standard" set of training data, so anyone can include it in post training plus a matching benchmark, so they can certify how well it works