Post

The greatest danger from AI isn’t that it suddenly “decides to kill us.” I’m increasingly more concerned about the human systems we build around it: bad data, security vulnerabilities, poor oversight, misplaced trust, inadequate safeguards and failures to verify before acting.

Minab is a horrifying example. The Iranian school site struck by the U.S. was classified in a military database as an IRGC facility, despite evidence that its use had changed. An analyst documented changes as early as 2019 but in a system that wasn’t connected to the primary targeting database.

Humans are putting increasingly powerful technology into systems with bad information, vulnerabilities and imperfect oversight then trusting the technology enough that nobody catches the error before the consequences become irreversible.