Post

"Never hallucinate or make anything up." yes, you can just demand that the LLM not make errors that's definitely how the technology works

There's actually evidence that this reduces hallucinations/errors. arxiv.org/abs/2304.13734 gist.github.com/yoavg/4e4b48...

That's not an internal state

What do you mean?

An internal state requires an internality.

Hidden layers aren't inside anything, they're just layers.

Ok but why are you telling me this? My point was that prompting "Never hallucinate or make anything up" actually helps. I never made any claims about hidden layers being inside of anything. I know how NNs work...

I'm saying that the framing of that paper anthropomorphises them.

Internal state just a technical term that refers to hidden layer activations, it has nothing to do with humans.

The field should choose less anthropomorphising technical terms.

The paper also talks about the model having an "internal notion". Is notion also an industry term?

Besides, the paper you posted doesn't show that this kind of prompting would work. That is a leap that requires the model acting on the state in a self-aware fashion, which would be internality. The kind of internality that the model does not have.

The way to show that putting such things in the prompt affects the output in a certain way would be to benchmark the prompt. You cannot infer it working that way from this finding without implicitly granting LLMs self-awareness.