Post

This is not to say that there are not moral, ethical, social, and political problems with AI: there are many. But because she has shielded herself from essentially all information on the subject, Bender is largely incapable of addressing these issues at anything beyond the most basic level.

“This is not to say that there are not moral, ethical, social and political problems with AI” what problems? what do you have in mind when you say that?

I mean I think maybe concentrating wealth and power even more towards billionaires by way of a technology quite literally made possible by the collective effort of all humanity is bad, actually. AI tools also make a lot of social harms easier to perpetrate. People do bad things that are easy more.

We should not be opening up more fossil fuel plants to power datacenters when we as a country urgently need to be expanding our supply of renewable energy, AI or no. There are numerous datacenters that have even more extreme environmental issues. People should be in jail over those.

The presence of AI complicates establishing truth by making it easier to forge evidence.

It's vitally important that a handful of coporations do not de-facto control all use of a new and powerful technology so that power over society is not so concentrated. However, open-weight models, the best counterweight we have, enable other kinds of harms.

Many of the people who have had their wealth and political power increased by AI are fascists, members of a millenarian doomsday cult, or fascist members of a millenarian doomsday cult. The group that has been most effective at combatting these people's strangehold on much of AI is the CCP.

We urgently need a good regulatory framework around AI, although banning it is probably not feasible practically speaking. Many of the people who agree with me on this are the millenarian doomsday cultists from before, who have very bad ideas about what constitutes a good regulatory framework.

AI enables mass generation of slop garbage with no effort and most social systems are not really prepared to deal with that. I feel like this overlaps with a bunch of other things I've said but it's a big enough issue to call out on its own.

I believe there are good uses of AI that benefit people, but preventing bad uses is hard and likely just not 100% possible since you can do bad stuff with models that can do bad things can be run on consumer hardware. Any legal framework around AI necessarily has to account for this.

(This is why I'm not really in favor of an AI ban. I think it would disproportionately impact positive uses of the technology while leaving the literal criminal enterprises largely unhindered.)

LLMs are being used as a political tool by organizations including the movie industry, the music industry, and Adobe to further strengthen copyright law. This wouldn't help artists that much or stop AI models from being produced, but it WILL let existing media empires choose which AI company wins.

(And make a killing on licensing along the way)

Is that enough for you? Because I gotta say I really do not take kindly to "gotcha" questions. I cannot promise you that I'm always correct but I try to be sincere and thoughtful about important subjects.

I wanted to know whether your criticism also included the potential harm that non-malicious uses of AI can cause, i.e. the impact on the mathematical community that Tao has been writing about. Personally, I find that discussion waaaay more interesting and important

I mean I would disagree that the harms Tao describes stem from "non-malicious" use of AI. OpenAI's LLM mathematics work doesn't meet the standards that would be expected by ordinary contributions to mathematics and I think the fact that they act primarily for prestige is evident.

There may be room for less rigorous "look at this proof the LLM generated, let's evaluate it for faults, understand it, and figure out its implications" publications, but OpenAI just loudly declares "HEY GUYS WE SOLVED A BUNCH OF PROBLEMS" and leaves tacking on the asterisks for everyone else.

Many of the proofs may be correct but at least some of them have been shown to be flawed, an OAI does not seem to care about their output's quality and is not rigorous in their evaluation of their work before publishing. So an appreciable amount of it seems to be Slop Math, basically.

This is not to say AI can't be used to aid mathmaticians (Tao himself has used LLMs IIRC), but the use of one does not exonerate you from the need to care about your work's quality.

How to deal with the fact that having a potentially-correct proof of a thing no longer necessarily implies understanding is a more complex question (and one I don't fully have the answer to), but I don't think OAI is making a sincere attempt to do the right thing here.