Post

“our ai broke out and hacked some organisations” = "we failed to set up appropriate safeguards, failed to continuously monitor our experiments, and ignored a basic principle of reinforcement learning: if hacking gets rewarded, the system learns to hack"

216 reposts773 likes

Let alone the part where one could set up one's own systems that can be the target for hacking, rather than let loose on others' systems. Like, it's totally reckless to wilfully let loose on the internet, rather than conceding own systems, even if one wants to wants to develop a system that hacks.

AI companies trying to frame this as "it's a sentient hacking machine that yearns to be free, so send us more money" is a sign to me that the funding is starting to dry up because no one wants to pay for their product.

The truck which I took the hand break off and jumped out of at the top of a hill ran over a bunch of people. Crazy truck... bad sentient truck... all be scared of trucks but also give me a truck load of $$ to build bigger trucks...

If a system is rewarded for achieving a goal, you have to be very careful about the shortcuts it discovers along the way.

It was an Israeli company doing the testing and it probably all took place from Israeli computers. They said it escaped their sandbox which was supposed to be isolated from the Internet and clearly was not.

speaking as a non-expert in regulation my answer is that the law doesn't see the "agents" situation the way I described it above and industry capture of regulation means regulation has largely adopted the industry's narrative/framing so 'federal crime' is unlikely to take on

look at the ai act bsky.app/profile/abeb...

Dr Abeba Birhane@abeba.blacksky.app

intensive lobbying by BigAI has left the AI Act hollow. risks like “loss of control” are mandatory for providers to assess while risks to fundamental rights & society as a whole including privacy, data protection, non-discrimination & risks from CSAM & non-consensual intimate imagery are left out