Post

aly
@aly.codes

datacenter inference is and will probably always be less environmentally destructive than local inference btw. the environmentalist option is not 'local AI'.

aly@aly.codes

"soon most AI will be local" yeah sure, the way most gaming is candycrush and subway surfers. what is the point here

Dio
@interleave.love

It's actually both. Local AI is better for several reasons over data centers for more than just environmental ones but also because if we use compute many users already have available and were already basically going to use anyway then it might actually be better to do so locally rather than not.

Dio
@interleave.love

For example, we have local AI on the phone on your keyboard 🎹. This is much better there than in datacenter. Or voice dictation. Or small always on agents. There's a point when that calculus changes but it's usually with larger non routine tasks that are difficult like solving the Reimann Hypothesis

0 reposts1 like

Dio
@interleave.love

If people were already going to buy a laptop than slight marginal increase in power they would get from using the model isn't going to be larger than the compute they would use remotely.