Post

Catastrophic failures are rare with frontier models now. People will get used to this and take fewer precautions. But I'd argue that, as these systems become ubiquitous and more powerful, we should actually take more. This morning, Claude happily deleted all the SSH keys on one machine.

Of course, it was a machine dedicated to Claude, I had a full backup, and the keys were passphrase-protected, so it wasn't a big deal. I've had interns do way worse. I've done way worse myself.

I wiped an entire production server with a PHP 3 script at my first job. My mentor was really nice about it. The price of learning. And I learned. But consider that these AI systems can perform a thousand actions per second, with an error rate of just 0.2%. That's two fuck-ups per second.