Post

Google managed to make the mobile versions of Gemma 4 E2B and E4B remarkably small, I'll have to look into how they manage this

I’m expecting that the new Siri is going to be powered by Gemma 4 as a base. Google has some good reasons of their own to develop on-device models of course. Still, I imagine we’ll see something like a Gemma-based on-device model, with. A Gemini-based fallback in the cloud for tougher queries.

local processing is likely to be faster either. It probably takes at least a second to load the model, and TPS is going to be lower locally.

I think it’s more ideology than efficiency in Apple’s case. Given so much of their brand positioning is on privacy, and on-device processing I’m expecting to continue in that vein. It’ll be interesting to see if there’s any tricks that Apple can play to manage memory and model latency.

Good point on identity, and it is true that Apple has focused heavily on local inference so far. I'm still quite skeptical, I suppose we could find out soon with their event coming up.