Pinned
A little bit late, but this paper has now been published in ACL. Note: more recent (unpublished) analyses suggest agents are getting better at searching the web—which I've been mentioning in my conference presentations this summer.
🚨 New working paper 🚨 Can LLMs with reasoning + web search reliably fact-check political claims? We evaluated 15 models from OpenAI, Google, Meta, and DeepSeek on 6,000+ PolitiFact claims (2007–2024). Short answer: Not reliably—unless you give them curated evidence. arxiv.org/abs/2511.18749