Post

Colin

@colin-fraser.net

A meta-thread of some threads that I have written on here

16/9/26, 10:43 pm0 reposts12 likes

Colin

@colin-fraser.net

16 Sept

Why people think AI will make us extinct.

Colin@colin-fraser.net

There are indeed thousands of LessWrong posts, as well as papers and books, which make a particular theoretical case for why AI will kill us all that many people find convincing. I'm going to try to explain it as faithfully as I can.

Colin

@colin-fraser.net

16 Sept

Why I think the preceding argument is wrong.

Colin@colin-fraser.net

Alright here’s what I think about this. For me, this argument is a little bit slippery about what it means for a system to have “a final goal”, and takes for granted that any intelligent system must behave as though it has such a thing. I don’t think this is necessary, and I don’t think LLMs do.

Colin

@colin-fraser.net

16 Sept

Shorter version of why I don't think LLMs will kill us all.

Colin@colin-fraser.net

LLMs won’t wipe out humanity because they just don’t have that dog in them.

Colin

@colin-fraser.net

16 Sept

An alternate view of RLHF

Colin@colin-fraser.net

I've been playing around with trying to get a better understanding of this, and I think I found a really interesting way to look at it using the dual form of the standard RLHF objective function. Somewhat math-heavy crank thread to follow.