Post

Chrome shipped an LLM Prompt API to the web platform. At Mozilla, we oppose this API. Here's why:

@jakearchibald.com - "In Chrome when you create a LanguageModel it will either use the Google Cloud API or download a 4GB model to run locally". We don't use the Google Cloud API. If the model isn't available we will kick off a download, but the API just doesn't work without the modal locally.

is there a permission check on the users machine before downloading? is it possible that end users get llms downloaded to their machines without their knowledge?

WASM based LLM inference in the browser is possible for a while. Those often download models in that size. Any website can download up to ~10GB of data in localstorage IIRC, if enough space on disk is available. Or other blobs, like e.g. astronomical images than can easily be several GB in size.