Post

A lot of work on the impacts of AI on educational performance so far are experimental. They randomize student access to some specialized AI learning tool and often find positive results. But much of student AI use (esp where we expect negative learning effects) is not so controlled.

12/5/26, 6:36 pm0 reposts8 likes

@precatlady.bsky.social

This strikes me as being a paper pretty uninformed about categorizing and measuring learning, then concludes with "AI neutral or good, overall", which makes me skeptical of the actual purpose of this exercise

@precatlady.bsky.social

I admit I'm being a miser in this case because I also find it odd that it's considered an economics paper, but that could be totally normal as I don't work in science and technology studies or economics or whatnot (anthropologist)

I make use of the "summer dip" in ChatGPT usage that was observed in 2023 and 2024, where ChatGPT traffic fell way off when school let out. In areas with especially big dips (LOTS of student usage) did test scores drop?

Doty

@corbindoty.bsky.social

13 May

Anecdotally, most ChatGPT use in high school seems to be by adults, particularly those in administrative roles. Though if students are presented with an AI slop school assignment, they will slop right back.

@ruoshuiresearch.bsky.social

Nope! Measuring aggregate test scores at the metro level in the US, I find no relationship between usage and test scores, across different grades in high school or subjects, or at different parts of the test score distribution.

Ruo Shui

13 May

would be interesting to see AP exams broken out

This implies that, at least as of 2024, student-led ChatGPT usage had basically no impact on test scores for high school students, or at least not enough of an impact to show up in aggregate statistics. This is despite roughly 80% of high school students reporting they use it at the time.