Post

OpenAI guy deeply concerned that OpenAI product is extremely powerful. no no this is not an ad. don't say this is an ad. it's The News

Eric Lipton NYT@ericlipton.nytimes.com

OpenAI’s top scientist, in a blog post on Sunday, warns that artificial intelligence is evolving so rapidly that it is becoming increasingly difficult for humans to understand and control it. Here is the post: openai.com/index/an-ali... And a couple of small excerpts--clear warnings.

222 reposts1.4K likes

Do you sincerely believe Bill Gates and the AI company execs saying this stuff are all fucking around and actually no AI isn’t powerful at all and isn’t a potential security threat? It’s all just a ploy to make ppl think AI is soooo smart and powerful?

If you believe you are building a system capable of causing monumental harm, and you continue to build that system, you should be prosecuted as though that system caused that harm, whether or not it's actually capable of it.

Like if I keep genuinely trying to build bombs, and I keep writing Zodiac-style to the authorities about how many people my bombs will kill, but my bombs never kill anyone because I'm an idiot who built them out of styrofoam and pipe cleaners... I still feel like I've committed some sort of crime.

*Eric opens Instagram and sees an ad saying "We're sorry. We made our shoes simply too good. We can never make amends for how high quality our product is."* My god, we were not prepared for this power.

It’s a pretty legit concern around cyber security. If you read the METR report, it becomes pretty clear we’re entering a new era.

METR@metr.org

METR and Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.

OK, but if it found a way to circumvent the sandbox then it wasn't a secure sandbox, and also not an impossible task, correct? Also, why tell this agent to complete an impossible task if they weren't looking for it to do precisely this?

Again where the failure to hold people accountable in 2008 comes back to bite us. If there was an inkling people could be held liable for what they unleashing they would at least pause before being reckless.