Post

Chrome shipped an LLM Prompt API to the web platform. At Mozilla, we oppose this API. Here's why:

@paul.kinlan.me

@jakearchibald.com - "In Chrome when you create a LanguageModel it will either use the Google Cloud API or download a 4GB model to run locally". We don't use the Google Cloud API. If the model isn't available we will kick off a download, but the API just doesn't work without the modal locally.

Michael Warren

@michaelwarren.dev

19 May

is there a permission check on the users machine before downloading? is it possible that end users get llms downloaded to their machines without their knowledge?

fry69

@fry69.dev

WASM based LLM inference in the browser is possible for a while. Those often download models in that size. Any website can download up to ~10GB of data in localstorage IIRC, if enough space on disk is available. Or other blobs, like e.g. astronomical images than can easily be several GB in size.

19/5/26, 7:56 am0 reposts0 likes

fry69

@fry69.dev

19 May

Or is the problem that the blob contains weights for an LLM?

Michael Warren

@michaelwarren.dev

19 May

i guess it’s not much different than wasm. but from what i know of google literally everything from users goes into the ad machine. i would assume prompts text would also. possibly along with training google models etc. feels like there should be a permission check around llms in general

fry69

@fry69.dev

19 May

> i would assume prompts text would also Current implementation and specs do not show any hint of this behavior. LLM inference happens locally, nothing gets sent to the could.