Post

Our generative AI guidance says AI sometimes generates inaccurate or fabricated responses to queries, an effect AI scientists call hallucination. If using the term hallucination, describe it as an issue associated with the technology that produces falsehoods or inaccurate or illogical information.

Some in the field prefer the term confabulation or simpler terms to describe the inaccuracies that don't draw comparisons with human mental illness.

What do you think? How would you describe it when AI creates inaccurate information?

9 reposts34 likes

Counterpointing myself, but the chief objection to “hallucinate” is that the term ascribes agency to a machine So I’m not sure any different verb would do the trick It might take more than just one word change is what I’m thinking 🤔

Maybe instead of a special term for when AI gets something wrong, we should have a special term when it gets something right I propose "lucky guess"

LLMs neither have mental states nor the ability to evaluate truth, so any terms like "hallucinate" that imply they do are inappropriate. Some suggestions: Verb: to mangle, to garble, to fail, to error out, to gibber, to bullshit Noun: error, nonsense, gibberish, failure, bullshit

Examples: LLMs are prone to fail, generating gibberish. When the chatbot said you were the president of Algeria, it was erroring out. I had ChatGPT write the reference for you, but it mangled the text and filled it with nonsense.

I would have said "confabulation" except that (surprisingly, to me) the leading synonyms for that seem to be like "discussion". After a while I decided I liked "fabrication" best. I agree that "hallucination" seems to imply conscious experience.

I think fabrication is great. All generative AI content is constructed (fabricated) independently of reality, since it's just based on what other people have said which may or may not be correct. And I think a fabrication (vs a lie) is a good way of describing that it's not corresponding to reality.

"AI sometimes generates fabricated responses to queries. AI scientists call these falsehoods "hallucinations," but mental health advocates suggest avoiding the analogy to human mental illness. Use the term "hallucinations" when quoting an AI scientist, but otherwise use "fabricated responses."

I'd say it's a reflection of the state of our society, when that same society will swallow hook line and sinker something that is flagrantly false, so long as it conforms with their worldview.

I am a real noodge when it comes to word choices. One way I have been able to rein myself in a little is a challenge I came up with: I don't get to complain about someone else's word choice unless I can suggest a replacement. Interested to hear ideas from @susanseager.bsky.social.

Perhaps 'produces nonsense'? The important part is to avoid suggesting LLMs* have consciousness, understanding or agency. *Ideally style guides should also require specificity regarding what technologies are being discussed and proscribe uncritical reproduction of the vague advertising term 'AI'.

If a calculator told you 2+2 = 5, you'd call it an error. If a GPS told you to turn onto a road that doesn't exist, it would be an error. When AI makes up inaccurate or just plain wrong information, guess what? That's an error. Calling it a "hallucination" humanizes a technological mistake.

Depends entirely on whether the model simply guessed wrong, or its train of thought can be shown to involve intentional misdirection. Nuance is required

What would you call it if the same words were found in a printed book? If it came from a database we'd call it "corrupt." The only difference with an AI is that the AI lacks reproducibility.

Output and erroneously output? These other words give it agency like it thinks when it, or the language ones at least, is basically souped-up meaningless autocompletion. It’s not lying, hallucinating, writing, stating or actively doing anything. Anthropomorphizing AI is disastrous.

Miscalculation? Model drift or prediction error aren't single words but are accepted alternatives. Confabulation is maybe a bit better, but these are alternatives.