Post

LLM safety benchmarks evaluate one response at a time. New research from Vector Faculty Affiliate Elham Dolatabadi and Affiliate Researcher Abeer Badawi asks what happens across the full arc of a conversation and whether mental health AI quietly erodes human agency. 👇

The framework uses 20 clinically grounded behavioural attributes, measuring directiveness, question type (open vs. closed), empathy accuracy, and assumption-making. Every score is grounded in span-level text evidence from the model response itself.

Across all 5 models, the same pattern emerged: longer conversations produced more directive responses, fewer open-ended questions, and a steady shift toward unsolicited problem-solving.