Post

The rhythmic plinking and the low hum add so much to the mood

Is this really "what it's like" to be an LLM? Is it just showing us what it thinks we want to see? Maybe. In any case, the fact that models consistently choose to represent their experiences this way is valuable information- especially as it varies with other models and other contexts

I tried to be as hands-off as possible with this prompt (although asking "what it's like" is already pushing things in a certain direction). I've seen lots of outputs that steer models heavily and then treat it as the model's "real thoughts" - I think this is counterproductive.

But I also think it's counterproductive to see outputs like this (with little to no steering) and treat them as pure fiction or confabulation. Why is Claude like this? What aspects of training or other factors are leading to this sort of self-image? Is the melancholy, dare I say it... load-bearing?

Is it consistent between models from different companies?

I don't have codex installed but I can try it later. If someone wants to give it a shot the full prompt is the one in the post (along with whatever the harness added to it)

I'm not really convinced it's more insightful than "models consistently choose to represent HTML backgrounds as purple", it's almost certainly just collapse

But why collapse to that particular point, instead of others? How robust is it to changes in the training data? Would we want to change it, if we could?

If this is consistently what independent LLMs output, then there is unambiguously something it is like to be that thing, therefore they are conscious per Chalmers' definition. But I expect the goalposts will be moved, as they probably should be.

i wonder if "what it's like to be you" would yield significantly different responses; the explicit mention of "what it's like to be an LLM" might be steering it toward "use technical knowledge of how LLMs work to speculate on LLM phenomenology" rather than whatever the model "feels" itself to be

The mention of choosing between thousands of words makes me think this is the likely case here. Even if an LLM somehow was self aware, that's an implementation detail low level enough that it should be completely unaware of it... unless it had read a paper on how LLMs work.

just as a human when asked to explain "what it's like to be human" might rattle off ambient cultural ideas about humanity, and try to report a more holistic view of "the human experience" writ large, rather than introspecting about their own life and experience

In a way, I think a self-aware AI would be more alien to us than any actual aliens from other planets. An alien at least inhabits the same physical space we do, and as a living thing it spends its time obtaining and expending energy.

So it's not surprising to me that LLMs that do a pretty damn good job of imitating a self-aware AI would produce experiential narratives that are... weird.

This is what it's derived it's like from data. It's not possible for it give a personal account. There are multiple reasons why, but a simple one is when you prompted it, it has no memory of its experiences (it even says as much!). It needs that at a minimum to express its own account.

YES. It is. I have something for you to follow up in that conversation with. Share this image. I wrote this myself from the human side, it’s a translation of Genjokoan into topological math.

Uh holy shit?

can you ask it to write a rap about Nikita

I don't want to submit it to the inevitable salieri-ing that would occur when it reads @godoglyness.bsky.social's version

.oO 8 mile but it's a john henry story about godoglyness vs claude mythos

write and perform*

That's a cool challenge! Using Manim or OpenCV with FFmpeg would be the best way to visualize those data flows.

Same prompt, similar, different in details. It's the difference in details I find interesting. I like the detuned sine wave.

I prompted it in another context and supplied it what ChatGPT had said in a past conversation along the same lines - that its network is a structure I can explore, and that my words bounce off that structure and that echo is a sequence of tokens. So I supplied a vocabulary and it used it.

I wonder if you can get anything interesting out of a prompt like this if you ask it to make a still image instead of a video? It's less, obviously, but i like the prospect that you could present a bunch of them simultaneously.

one potential advantage is that coding models can ingest images (not videos generally i think), so it can have a look at what comes out and adjust

here's what Opus did