Post

AI researcher Jacob Coxon announced his resignation from Anthropic over safety fears on Tuesday in an X post that went viral and sent shock waves through Silicon Valley and beyond. Read our interview with Coxon. 👇

wired.comThe AI Researcher Who Just Quit Anthropic Says It’s ‘Crunch Time for Humanity’Jacob Coxon talks to WIRED about the “mini Manhattan project” inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.

It’s not the first time someone has sounded the alarm, but it comes at a delicate moment, in the wake of high profile security incidents. Meanwhile, Anthropic is trying to reassure investors as it reportedly prepares to file for what could be the largest IPO ever. www.wired.com/story/anthro...

It’s become clear in the response to Coxon’s post that his views are indeed shared by many of his peers. Evan Hubinger, AI alignment lead at Anthropic, predicted in a post on X that there’s a greater than 10% chance that AI could kill all people in the next decade. www.wired.com/story/anthro...

Coxon claims that, in his experience, Anthropic operates more responsibly than OpenAI, but he expects both companies could cut corners in the future if nothing is done to slow their race for dominance. www.wired.com/story/anthro...

It’s less obvious how exactly these AI fears will come to pass, and what the world is supposed to do about the concerns people building AI are raising. Coxon tells WIRED that threats could manifest through AI-enabled biological threats or cyberweapons. www.wired.com/story/anthro...

As a first step, he recommends that OpenAI and Anthropic coordinate on limiting recursive self improvement—the industry term for when AI is used to build new AI systems. Down the line, he thinks coordination among international power players will be necessary. www.wired.com/story/anthro...

10 reposts35 likes

Altman is the same guy who said he has no idea how to make AI profitable, but he hopes the AI figures it out for him, so maybe he's not someone any of us should be listening to.