Post

this is why actually regulating datacenters is a lot more important than banning them. banning them is stupid. please just set reasonable floors for % renewables!

fwiw, I am not saying no to local AI there are a million reasons why local AI is desirable in some capacity. I am an ops freak and love making my computer fan go spinny running a model it has no business running. I want my phone's features to work offline. I don't like daddy jensen etc.

but you guys are actually deluding yourselves. like you are just not correct on the face of it.

It's actually both. Local AI is better for several reasons over data centers for more than just environmental ones but also because if we use compute many users already have available and were already basically going to use anyway then it might actually be better to do so locally rather than not.

For example, we have local AI on the phone on your keyboard 🎹. This is much better there than in datacenter. Or voice dictation. Or small always on agents. There's a point when that calculus changes but it's usually with larger non routine tasks that are difficult like solving the Reimann Hypothesis

hm, is this to do with e-waste concerns?

not just ewaste but energy sourcing + distribution is worse to residential lines, consumer batteries are certainly worse, and work cannot be pooled for efficiency gains (e.g. your framework desktop has a floor in wattage even when you're not using it that is basically wasted)

datacenters are more efficient for computer generally

tbh i can't wait until gpus have more specialized hardware for inference (we have compute cores yes but i'm thinking about something like a tpu in your gpu) so that you can run small models for translation, accessibility features (better ocr screen readers), transcription, etc

i know that a lot of modern cpus have these but afaict these have basically zero software support

I want local AI for the sane reason I ride my motorcycle, and I fawn over datacenfer hardware and support its proliferation for the same reasons as my train 'tism

yeah this is pretty much efficiency / energy / sustainability 101 and true of ...many things. like the issue with all the big conglomerates is more down to policy and biz and them choosing to not do better, not that they're necessarily worse environmentally. that said, how we get to a better place

Idk. Our grid in Aotearoa is mostly renewable and when I run local models that's what I pull from. As well no industrial cooling n I'm trying to just get it on mini solar. But it's more about data sovereignty than diseconomies of scale imo.

I mean sure but you're also not getting nearly the efficiency of just throwing 0.001% of a hyperscalar at a problem for 30 seconds with a much bigger model. yeah it's great it's all renewable but that doesn't change the efficiency equation

Efficiency is an economy of scale question which I think is pretty settled in yr favour but just on the net environmentally destructive question I feel like there's some nuance to be had. Esp in the mesh pooling and co core era post u biquitous home solar.