Post

A NeurIPS submission got caught with hallucinated citations. The authors' admitted they asked ChatGPT to fill in BibTeX from paraphrased in-text citations. We ran a simulation to see how often this fails, and built a free tool to catch it. #AcademicSky

First, the tool: the Paperpile Citation Checker. Upload your BibTeX, and it verifies every reference in real time. When it's done, you can download a cleaned-up file or share the results URL with coauthors.

Why now? This month alone, large-scale analyses in The Lancet and on arXiv highlighted how widespread hallucinated citations have become. And arXiv announced a severe penalty for submitting manuscripts with hallucinations.

How does this happen? A case study on OpenReview spells it out: authors gave ChatGPT paraphrased citations (author-year in-text citations, or titles) and asked it to generate properly formatted BibTeX. Instead of looking up the real metadata, the model hallucinated it.

We also found a strong correlation between citation count and hallucination risk. Highly-cited papers appear more often in training data, so models get them right. But newer and less-cited references tend to be more prone to AI-induced hallucinations.

We spot-checked today's top AI assistants on the same task. ChatGPT and Claude were relatively clean. Gemini had more hallucinations. The difference? ChatGPT and Claude ran dozens of web searches while generating the output. Web search is what grounds these tools.