Post

LLM safety benchmarks evaluate one response at a time. New research from Vector Faculty Affiliate Elham Dolatabadi and Affiliate Researcher Abeer Badawi asks what happens across the full arc of a conversation and whether mental health AI quietly erodes human agency. 👇

The benchmark, Cognitive Atrophy Bench, is built on 1,576 real counselling conversations (not synthetic prompts), 15,680 dialogue turns and 42,230 responses across 5 frontier LLMs, and reviewed by clinical psychology experts. Real data with rigorous annotation.

Across all 5 models, the same pattern emerged: longer conversations produced more directive responses, fewer open-ended questions, and a steady shift toward unsolicited problem-solving.