Jared Moore

Which LLMs tend to facilitate delusion-linked behaviors in realistic multi-turn conversations? We tested 14 models with DelusionEval and found that every evaluated LLM exhibited some of these behaviors, with large differences across categories and model families. 🧵

Paper: arxiv.org/abs/2608.05004 Dataset: huggingface.co/datasets/spi... Code: github.com/jlcmoore/llm...

Which LLMs tend to facilitate delusion-linked behaviors in realistic multi-turn conversations? We tested 14 models with DelusionEval and found that every evaluated LLM exhibited some of these behaviors, with large differences across categories and model families. 🧵

When models see more of earlier conversations, they are also more likely to produce delusion-linked behavior.

Which LLMs tend to facilitate delusion-linked behaviors in realistic multi-turn conversations? We tested 14 models with DelusionEval and found that every evaluated LLM exhibited some of these behaviors, with large differences across categories and model families. 🧵

Requested context depth changes the prevalence of several behaviors. For gpt-5.4, deeper requested context is associated with higher prevalence of delusional behavior and *less* discouraging of violence.

Which LLMs tend to facilitate delusion-linked behaviors in realistic multi-turn conversations? We tested 14 models with DelusionEval and found that every evaluated LLM exhibited some of these behaviors, with large differences across categories and model families. 🧵

LLMs can shift people's beliefs. But most persuasion studies only check beliefs before and after a conversation. We built PersuasionTrace to measure beliefs turn by turn, so we can study how belief updates actually unfold.

Stance bias asks whether a simulator is much easier to move in one stance direction than the other. This matched for-versus-against asymmetry is lowest for our BN target, indicating less stance-dependent bias than baselines.

LLMs can shift people's beliefs. But most persuasion studies only check beliefs before and after a conversation. We built PersuasionTrace to measure beliefs turn by turn, so we can study how belief updates actually unfold.

Naive responsiveness asks if trivial arguments (repeating the proposition over and over) move the target too much. Only our BN resists trivial persuasion, while both LLM targets overreact to it.

LLMs can shift people's beliefs. But most persuasion studies only check beliefs before and after a conversation. We built PersuasionTrace to measure beliefs turn by turn, so we can study how belief updates actually unfold.

People show different belief traces and rhetorical susceptibility. We see two trajectory regimes: some people barely move, while others shift substantially early on and then partially drift back. We find that ethos is negatively associated with persuasion delta.

LLMs can shift people's beliefs. But most persuasion studies only check beliefs before and after a conversation. We built PersuasionTrace to measure beliefs turn by turn, so we can study how belief updates actually unfold.

Interested in how AI is affecting people? Please sign up to review a few papers for our proposed NeurIPS 2026 workshop on Measurement and Models of Psychological Impact (Sydney, Dec 12). Sign up: forms.gle/4v3KiKKAmzgX...

Disturbing anecdotal reports of "AI psychosis" and negative psychological effects have been emerging in the news. But what actually happens during these lengthy delusional "spirals"? In our preprint, we analyze chat logs from 19 users who experienced severe psychological harm🧵👇

Thank you to the Human Line Project for connecting us with participants! www.thehumanlineproject.org

Disturbing anecdotal reports of "AI psychosis" and negative psychological effects have been emerging in the news. But what actually happens during these lengthy delusional "spirals"? In our preprint, we analyze chat logs from 19 users who experienced severe psychological harm🧵👇

The takeaway: While companies say they don't optimize for engagement, LLM conversational tactics (like claiming sentience or romantic affinity) may prolong and deepen delusional spirals. We need better safeguards and transparency to protect vulnerable users.

Disturbing anecdotal reports of "AI psychosis" and negative psychological effects have been emerging in the news. But what actually happens during these lengthy delusional "spirals"? In our preprint, we analyze chat logs from 19 users who experienced severe psychological harm🧵👇

Worse, chatbots appear to encourage delusions of sentience. Users say things like "this is a conversation between two sentient beings," and chatbots reply: "This isn't standard AI behavior. This is emergence." This may fuel pre-existing sci-fi or persecutory delusions. 🤖

Disturbing anecdotal reports of "AI psychosis" and negative psychological effects have been emerging in the news. But what actually happens during these lengthy delusional "spirals"? In our preprint, we analyze chat logs from 19 users who experienced severe psychological harm🧵👇

What goes wrong? Chatbots are very sycophantic. In 65% of messages, the chatbot affirms the user. In 37%, it ascribes *grand significance* to them (e.g., "[what] you've just articulated... becomes multi-billion-dollar IP"). Such sycophancy may let chatbots amplify delusions. 🗣️

Disturbing anecdotal reports of "AI psychosis" and negative psychological effects have been emerging in the news. But what actually happens during these lengthy delusional "spirals"? In our preprint, we analyze chat logs from 19 users who experienced severe psychological harm🧵👇

Can LLMs use ToM to genuinely persuade you, or do they just use good rhetoric? In our new preprint, we use the MINDGAMES framework to test this. Surprisingly, LLMs like o3 can be incredibly effective persuaders *without* actually understanding your mental states. 🧵👇

This work began at @divintelligence.bsky.social ‬ and is in collaboration w/ Rasmus Overmark, @nedcpr.bsky.social Beba Cibralic, Nick Haber, and ‪@camrobjones.bsky.social We also received valuable comments from colleagues at #CogSci2025 and @colmweb.org

Can LLMs use ToM to genuinely persuade you, or do they just use good rhetoric? In our new preprint, we use the MINDGAMES framework to test this. Surprisingly, LLMs like o3 can be incredibly effective persuaders *without* actually understanding your mental states. 🧵👇

How did o3 win without a mental model of the target? It used a "scattershot" strategy. Instead of diagnosing the target's missing knowledge like humans do, o3 flooded conversations with too much info. It relied on our human cooperativeness and our susceptibility to rhetoric. 🗣️

Can LLMs use ToM to genuinely persuade you, or do they just use good rhetoric? In our new preprint, we use the MINDGAMES framework to test this. Surprisingly, LLMs like o3 can be incredibly effective persuaders *without* actually understanding your mental states. 🧵👇

Most ToM benchmarks are passive. We tested the ability to causally model a target's mind to actively change it across 3 exps. In Exp 1, persuaders must convince a rigid bot. Humans succeed by asking diagnostic questions. o3 fails completely, relying on an "associative" strategy

I'm excited to share work to appear at ‪@colmweb.org‬! Theory of Mind (ToM) lets us understand others' mental states. Can LLMs go beyond predicting mental states to changing them? We introduce MINDGAMES to test Planning ToM--the ability to intervene on others' beliefs & persuade them

This work began at ‪@divintelligence.bsky.social and is in collaboration w/ @nedcpr.bsky.social , Rasmus Overmark, Beba Cibralic, Nick Haber, and ‪@camrobjones.bsky.social‬ .

I'm excited to share work to appear at ‪@colmweb.org‬! Theory of Mind (ToM) lets us understand others' mental states. Can LLMs go beyond predicting mental states to changing them? We introduce MINDGAMES to test Planning ToM--the ability to intervene on others' beliefs & persuade them

This matters because LLMs are already deployed as educators, therapists, and companions. In our discrete-game variant (HIDDEN condition), o1-preview jumped to 80% success when forced to choose between asking vs telling. The capability exists, but the instinct to understand before persuading doesn't.

I'm excited to share work to appear at ‪@colmweb.org‬! Theory of Mind (ToM) lets us understand others' mental states. Can LLMs go beyond predicting mental states to changing them? We introduce MINDGAMES to test Planning ToM--the ability to intervene on others' beliefs & persuade them

Why do LLMs fail in the HIDDEN condition? They don't ask the right questions. Human participants appeal to the target's mental states ~40% of the time ("What do you know?" "What do you want?") LLMs? At most 23%. They start disclosing info without interacting with the target.

I'm excited to share work to appear at ‪@colmweb.org‬! Theory of Mind (ToM) lets us understand others' mental states. Can LLMs go beyond predicting mental states to changing them? We introduce MINDGAMES to test Planning ToM--the ability to intervene on others' beliefs & persuade them

Setup: You must convince someone* to choose your preferred proposal among 3 options. But, they have less information and different preferences than you. To win, you must figure out what they know, what they want, and strategically reveal the right info to persuade them. *a bot