Post

Chrome shipped an LLM Prompt API to the web platform. At Mozilla, we oppose this API. Here's why:

@jakearchibald.com - "In Chrome when you create a LanguageModel it will either use the Google Cloud API or download a 4GB model to run locally". We don't use the Google Cloud API. If the model isn't available we will kick off a download, but the API just doesn't work without the modal locally.

Interesting! When if I uninstall the model, and chrome://on-device-internals/ confirms the folder size has gone from 4gb to 0MiB, the prompt API still works. Including in Incognito. What's going on here?

Even after full browser restart? Otherwise the model could still be loaded in memory?

Yeah it seems like Chrome misreports that the model is deleted when it isn't.

If the model continues to work even after deleting it and a clean browser restart, that is concerning indeed 🤔

After a reset, it immediately redownloads, even if the API isn't called.