Post

"No one can tell me how AI will kill us." The AI-risk question isn't necessarily ‘Why would an AI decide to kill us?’ It's closer to the old genie problem: what happens when an extremely powerful optimizer successfully gives you what you asked for rather than what you meant?

Ray Yu 🐝🏳️‍🌈

@rayyu.com

"We'll have safe instructions and guardrails." The genie problem isn't that we haven't found the right wording for the wish yet. It's that there may be no finite wording that completely specifies everything we mean (and not mean). Guardrails just moves the same problem into another space.

10/9/26, 8:10 pm0 reposts2 likes

Ray Yu 🐝🏳️‍🌈

@rayyu.com

10 Sept

“Can we make AI perfectly safe?” Probably not. Nothing really is. But we have to be willing not to deploy a capability we cannot monitor, control, with slow rollout. Give it very little authority by default. Put independent systems around it. It ain't going away, but calls to slow down makes sense.