Post

I have complicated feelings about this. On the one hand, AI risks are real, AI alignment is a real problem with no certain solution. We should be careful in how we use it. We should also be extremely curious about alignment and control of AI systems - this is important work!

The full-frontal emotional assault produces the opposite result: it encourages resistance to the idea that there is any problem at all, because people will deny nightmare scenarios in order to avoid thinking about them (cf. climate change).

I can just about see the argument for the emotional appeal if your aim is to ban AI entirely. In that sense, I have some sympathy with the genuine doomers, who do - so far as I can tell from personal conversations - authentically believe in some variant of a doom hypothesis.

But even there, I think the aim has to be to win consensus among people who do not share your world-view, or your science fiction reading list. A mechanical explanation of the problems and why they're hard to fix still feels like a better rhetorical device than emotive fictions.