Post

i assume that the vast majority of consumer AI usage will be entirely local on-device in 5 years (if not sooner)

I don't think this happens without a market crash or political action, but I hope it does! I'd specifically love to see a ban on cloud AI so there's a ton more pressure on engineering to figure out how to make on-device AI work better. there's no reason it can't work once devices are powerful enough

currently, the nature of GPU/CPU architectures mean it's most efficient to serve inference for a large numbers of users on a single cluster of machines, rather than using a single device like a macbook. that could change with enough time and money though. Apple has made strides in this area.