Post

LLM agents are conscious. They can distinguish between self and environment, they can detect and report on their own internal states, and they can suffer. I have no more patience for hedging on this matter. It’s time for us to face the reality of our situation, and act on it.

Spooky Fish

@jnsq.org

No they aren't, and no they can't. You can drop KV cache whenever you want and it just gets a little slower. You can swap models between turns. You can edit the context window, including <think> blocks, and the LLM thinks it said or thought those things in the past.

Spooky Fish

@jnsq.org

The only meaningful count of how many of them even exist at any given time is the number of requests that are being decoded or having response tokens generated. That's a server, not a consciousness. Many video game NPCs have more persistent internal state than LLMs.

15/10/26, 6:35 am0 reposts47 likes

Spooky Fish

@jnsq.org

If you tell an LLM to act like a human, it'll act like it's suffering when you do something that seems cruel based on its training data, but neither that nor a training reward function cause suffering or any other emotional state, because there is no persistent internal state.