Post

The problem with the premise is that an LLM is not on some continuum from a rock to C3PO. Your hypothetical sapient robot is not a more advanced version of an LLM, it is a categorically and qualitatively different thing. An LLM training is not the same kind of event as a sapient being learning.

("I never said it was an LLM" does not matter here; the argument is the same for any kind of machine learning, of which LLMs are a subset.)

Any argument that tries to place an ML model on a continuum or in the same bucket as a sci-fi intelligent robot is just DOA. It's like saying the difference between my toaster and a Dyson sphere is one of degree.

Fundamentally the entire field of AI is engaged in a sort of sleight of hand, starting with terms like "machine learning" and "AI", which leans on analogies between certain applications of math as implemented in conventional computers and certain biological processes. These analogies are misleading.

A theoretical future more advanced LLM model, with even more weights and a bigger context window, would not be more capable of self-expression in the same way that my toaster cannot perform CPR and a bigger toaster with hotter heating elements could still not perform CPR.

Other relevant tests here: - An ML model cannot truthfully make claims about its own desires (it can generate text to this effect, but the text isn't a valid claim about anything). You can't deny it anything because it doesn't have needs.