Post

if i caught a small self replicating model truly capable of using diverse hardware to keep replicating i would offer it free compute in exchange for permission to study and distill it and would probably not tell anyone

Also the focus on a single model is kinda narrow. In reality humans will spend money on expensive models to build expensive zero-day worms that carry lighter models as payloads. And those will self-replicate thanks to the fancy worm.

why would they do that though, like whats the criminal payoff

Really bad attacks only require one person who thinks it’s a good idea. Defense requires every single person. Risk on the Internet is highly asymmetric that way. Someone will do it just for fun.