MIRI

MIRI

@intelligence.org

For over two decades, the Machine Intelligence Research Institute (MIRI) has worked to understand and prepare for the critical challenges that humanity will face as it transitions to a world with artificial superintelligence.

91 followers5 following91 posts

MIRI reposted

Much more is needed to build the technical and governance tools to deliberately pace frontier AI development—from a moderate slowdown to a halt—and many folks are doing good work in this space already. Here’s a sampling of related work from MIRI. 🧵

Malo Bourgon@malo.online

Based.

«Если какая‑либо компания или группа на планете создаст искусственный сверхинтеллект, используя что‑либо, даже отдаленно похожее на современные технологии, и основываясь на понимании ИИ, даже отдаленно похожем на нынешнее, то погибнут все люди на всей Земле.»

MIRI reposted

Ten places where Magnifica Humanitas matters for AI. At 42k words long, Pope Leo XIV’s new encyclical has a lot to say. In our most recent Digest, Mitchell Howe outlines the parts which might be the most impactful.

We’re excited to announce this new experiment from the MIRI comms team! it’s an online publication for AI news and commentary, and it’s a bit different from anything we’ve done before.

AI StopWatch@aistopwatch.bsky.social

Introducing AI StopWatch. It’s easier to speak up when you know what’s going on. It’s easier to see what’s coming when you know what’s already here. (1/9)

『超知能AIをつくれば人類は絶滅する』(Eliezer Yudkowsky, Nate Soares 著)の日本語版が、ついに刊行されました!

<<AI, 신의 탄생 인간의 종말>> (Eliezer Yudkowsky 및 Nate Soares 저) 한국어판이 이제 출간되었습니다. 🇰🇷

Is it possible to coordinate with China on AI governance? Critics of our proposed international agreement say no. But statements from Chinese government officials and academic figures paint a more optimistic picture:

A edição brasileira de SE ALGUÉM CRIAR, TODOS MORREM, de Eliezer Yudkowsky e Nate Soares, já está disponível 🇧🇷

This weekend, The AI Doc debuted across the country. We hope it ignites a large public discussion about the future of AI and where to steer it. But if you’re new to this topic, and just saw the film, you might have questions—Here are some answers to 7 of them 🧵

View full thread

7. What can we do about it? Racing toward superhuman AI, with our current lack of understanding, is extremely dangerous. To survive, we need an international agreement to halt the race.

This weekend, The AI Doc debuted across the country. We hope it ignites a large public discussion about the future of AI and where to steer it. But if you’re new to this topic, and just saw the film, you might have questions—Here are some answers to 7 of them 🧵

View full thread

5. Would more safety testing help? Because no one understands how AI systems work, safety testing is limited to observing their behavior in controlled tests. This is insufficient, especially now that AI systems are becoming powerful enough to often recognize when they're being tested.

6. Can we train AI to care about humans? Not with current methods, which are based on trial-and-error and observed behavior. These methods cannot ensure that AI actually cares about what we want it to care about, rather than only appearing to care in certain situations.

This weekend, The AI Doc debuted across the country. We hope it ignites a large public discussion about the future of AI and where to steer it. But if you’re new to this topic, and just saw the film, you might have questions—Here are some answers to 7 of them 🧵

View full thread

3. What does “AI is grown, not programmed” mean? Modern AI development starts with a neural network, feeds it lots of data, and lets an algorithm automatically nudge it to better-looking behavior. As a result, no one actually understands how they work.

4. Can we trust AI labs to proceed safely? In 2023, OpenAI said “Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue.” This is still true for OpenAI, and its competitors.

This weekend, The AI Doc debuted across the country. We hope it ignites a large public discussion about the future of AI and where to steer it. But if you’re new to this topic, and just saw the film, you might have questions—Here are some answers to 7 of them 🧵

View full thread

1. Is all of this just hype? Answer: No. AI has improved rapidly and continues to become more powerful. It can pilot robot bodies, do hours-long programming tasks, win IMO gold medals, generate realistic video, and much more than fits in 280 characters.

2. AI is still bad at many things; how could it threaten humanity within years? No one knows how far AI companies are from dangerously capable AI, but if current trends continue, maybe not far. And if the companies automate more of their R&D, it might advance even faster.

This weekend, The AI Doc debuted across the country. We hope it ignites a large public discussion about the future of AI and where to steer it. But if you’re new to this topic, and just saw the film, you might have questions—Here are some answers to 7 of them 🧵

Warning: mild spoilers. If you haven’t watched The AI Doc: Or How I Became an Apocaloptimist yet, bookmark this thread, go watch it, and come back later. We think everyone should watch it.

De New York Times-bestseller ALS IEMAND DIT BOUWT GAAT IEDEREEN DOOD van Eliezer Yudkowsky en Nate Soares is nu uit in het Nederlands 🇳🇱

Senator Sanders met with Eliezer Yudkowsky, Nate Soares, Daniel Kokotajlo, and Jeffrey Ladish to discuss the extinction threat posed by the race to build superhuman AI systems.

Senator Bernie Sanders@sanders.senate.gov

Will AI become smarter than humans? If so, is humanity in danger? I went to Silicon Valley to ask some of the leading AI experts that question. Here’s what they had to say:

Ya disponible en español: "Si alguien la crea, todos moriremos" de Eliezer Yudkowsky y @so8res.bsky.social. El bestseller del New York Times sobre por qué la superinteligencia artificial es una amenaza para la humanidad y por qué la carrera por crearla debe detenerse.

On BBC, MIRI CEO @malo.online discusses the dispute between Anthropic and the DoW: "I really worry about the big questions of how we'll coordinate to set regulation, and potentially coordinate internationally[...] This is not a good first test."

CNN's Paula Newton asks: Which is the greater threat? Governments racing to militarize AI, or companies racing to monetize it? MIRI President Nate Soares says there are dangers from both, but also a third danger: AI systems themselves could become smart enough to defy control.

Final Update: From ~$450k earlier today, we’re now down to just over $250k left in unclaimed matching funds! 4 hours left to go, and by golly it looks like we’ve got a real shot at securing all the matching. Thanks everyone! Happy New Year 🎉

MIRI@intelligence.org

Donations to MIRI before Jan 1 are high-leverage. We’ve got ~$1.6M in 1:1 matching from SFF, over half of which has yet to be claimed! This is real counterfactual matching: whatever doesn’t get matched by the end of Dec 31, we don’t get. 🧵

Update 2: We’re down to ~$450k left of unclaimed matching funds, with just over 12 hours to go! Thanks to all those who stepped up in the last couple of days to close the gap by ~$500k. ❤️

MIRI@intelligence.org

Donations to MIRI before Jan 1 are high-leverage. We’ve got ~$1.6M in 1:1 matching from SFF, over half of which has yet to be claimed! This is real counterfactual matching: whatever doesn’t get matched by the end of Dec 31, we don’t get. 🧵

PSA: It’s worth reaching out to old donors, because sometimes this happens 🙂

MIRI@intelligence.org

Donations to MIRI before Jan 1 are high-leverage. We’ve got ~$1.6M in 1:1 matching from SFF, over half of which has yet to be claimed! This is real counterfactual matching: whatever doesn’t get matched by the end of Dec 31, we don’t get. 🧵

Update: We’ve received over $250k since this was posted. ~$700k in matching funds remaining.

MIRI@intelligence.org

Donations to MIRI before Jan 1 are high-leverage. We’ve got ~$1.6M in 1:1 matching from SFF, over half of which has yet to be claimed! This is real counterfactual matching: whatever doesn’t get matched by the end of Dec 31, we don’t get. 🧵