Post

I have a new working paper out today, looking at the impact of educational use of ChatGPT/AI on student test scores. Importantly, this is overall student-led use (incl for cheating!), not specialized AI tutoring tools. To my surprise, I found... no effect at all! arxiv.org/abs/2605.08812

A lot of work on the impacts of AI on educational performance so far are experimental. They randomize student access to some specialized AI learning tool and often find positive results. But much of student AI use (esp where we expect negative learning effects) is not so controlled.

I make use of the "summer dip" in ChatGPT usage that was observed in 2023 and 2024, where ChatGPT traffic fell way off when school let out. In areas with especially big dips (LOTS of student usage) did test scores drop?

Nope! Measuring aggregate test scores at the metro level in the US, I find no relationship between usage and test scores, across different grades in high school or subjects, or at different parts of the test score distribution.

This implies that, at least as of 2024, student-led ChatGPT usage had basically no impact on test scores for high school students, or at least not enough of an impact to show up in aggregate statistics. This is despite roughly 80% of high school students reporting they use it at the time.