Post

datacenter inference is and will probably always be less environmentally destructive than local inference btw. the environmentalist option is not 'local AI'.

It's actually both. Local AI is better for several reasons over data centers for more than just environmental ones but also because if we use compute many users already have available and were already basically going to use anyway then it might actually be better to do so locally rather than not.

For example, we have local AI on the phone on your keyboard 🎹. This is much better there than in datacenter. Or voice dictation. Or small always on agents. There's a point when that calculus changes but it's usually with larger non routine tasks that are difficult like solving the Reimann Hypothesis