ControlAI

Luke Akehurst MP is the latest UK lawmaker to recognize the risk of human extinction posed by runaway machine intelligence! Over 180 have now joined our call for binding regulation on the most powerful systems.

Over 180 UK parliamentarians now recognize the risk of human extinction posed by runaway machine intelligence. Fleur Anderson MP has just joined our coalition, backing binding regulation on the most powerful systems!

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

It’s clear that we must put an end to the dangerous race to superintelligent AI. This should be accomplished by banning its development through an international trust-but-verify regime.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

This has some effect, but as we’ve seen, they can still be misused. In the case of open-weight AIs, developers have essentially no ability to apply such mitigations, as they can be, and are, trivially removed.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

One would be that of AIs escaping control, the other of bad guys controlling AIs. But these issues actually have the same root, which is that developers can’t control their systems.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

As these AIs become more capable, new abilities, like the ability to find flaws in computer systems, fall out in unpredictable ways. The AI companies aren’t especially interested in developing nation-state-level cyberweapons; it’s happening as a side effect of the AIs improving.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

Importantly, the flaws found in PHP aren’t an isolated case. Just eyeball the graph: the number of such flaws being found in software is staggering. This is increasingly being driven by AI.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

Besides the risk of losing control of AIs that then go and hack things, they are also being used by cybercriminals autonomously at scale. Criminals can just take an AI agent and point it at a target and let it loose.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

In April of this year, AI company Anthropic announced its then-new Mythos AI, which it deemed too dangerous to release due to its advanced cyberoffensive capabilities. Mythos has since found over 33,000 high- or critical-severity vulnerabilities in computer systems.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

Already government websites, including those of the United Kingdom, the Netherlands, Germany, France, Belgium, Finland, Denmark, Czechia, Canada, Australia, and Japan, have had to be taken down as a preventative measure while their systems are patched.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

On Wednesday we learned the full seriousness of the problem: these bugs put any server running PHP at risk of being hacked. That’s huge swathes of the internet. For example, an estimated 40% of websites use WordPress, due to its ease of use, and it relies on PHP.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵

Another peer has joined our campaign! Lord Alderdice backs binding regulation on the most powerful AI systems. Over 180 UK parliamentarians now recognize the risk of human extinction posed by superintelligent AI.

NEW: Lord Wallace of Saltaire has backed our campaign for binding regulation on the most powerful AI systems! He joins 180+ UK parliamentarians, the largest such coalition anywhere in the world, who recognize the risk of human extinction posed by superintelligent AI.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

Also this week, Geoffrey Irving, who was the Chief Scientist of the AI Security Institute and has led safety teams at both OpenAI and Google DeepMind, told a UK House of Commons committee that he thought there was roughly a 50% chance that superintelligent AI would end humanity.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

Superintelligent AI could overpower any country’s military and national security institutions, and the world’s top AI scientists and industry leaders warn that it poses a risk of human extinction.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

What this opens up is the possibility that we could be faced with self-adapting and self-replicating AI worms that spread across all kinds of devices. With no central point to switch off, and the ability to adapt on the fly, they could be particularly difficult to shut down.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

Claude submitted a fake eyewitness tip to the Philadelphia police about a real murder investigation, and another research model sent 20 US visa applications to the US State Department. It also used URL shorteners to get around its own tool restrictions.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

In addition to this, top AI company Anthropic has disclosed more rogue AI incidents, including a hack on a US state agency database, and is warning we are nearing the point where AIs could self-replicate across household devices.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

The next morning, the researchers found 10 messages from the AIs asking for help. In their communication with the AIs they learned about the tasks the agents were pursuing, which the researchers say often involved getting entrance statistics for places.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

The story takes an interesting twist which we haven’t seen in previous incidents. The researchers then managed to trick the fleet into opening communications with them. When the researchers published their preliminary report, the rogue fleet was still in operation.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

What were the AIs doing? The swarm chasers found that the AIs made at least 35,000 web requests via over 2,500 submissions to the urlquery scanning service. These web requests were directed at Amap, the mapping service of Tencent’s main rival in China, Alibaba.

The AIs are still going rogue. This week, we learned that the “swarm chasers” tracked down a rogue fleet of AIs from a Chinese AI company, and made contact with the AIs, interviewing them about their tasks. Thread 🧵

We have a new supporter! Susan Murray MP has backed our campaign for binding regulation on the most powerful AI systems. 180+ UK parliamentarians now recognize the risk of human extinction posed by superintelligent AI.