Post

This study used a platform called Prolific, where people sign up to get paid small fees to do AI surveys, & they're incentivized to do them rapidly so they can complete more surveys to get more payments. These studies using Prolific have *very* questionable methodology & I would not read into them.

For example "A record spun lazily in the corner, the soft crackles of the needle the only sound filling the silence." That sentence doesn't make sense (isn't there music coming out of the record?), but if you're just reading as quickly as possible to finish the task and get paid you might miss that.

What's depressing, and what the study reveals, is that the vast majority of readers aren't paying much attention & don't have much discernment.

I'm old enough to remember that that's what it sounded like when the record reached its end, and the needle is just going around on the innermost groove (and there's often a specific crackle each time the needle reaches the point where the spiral groove reaches that innermost circle.

I'd be curious about a follow up study where the population is specific. Software engineers spend a lot of time engaging with AI authored text now and I can feel a distinct difference when it's used. But I don't know if that transfers generally to all types of writing.

I just read a story that was good, but I genuinely couldn't tell if it was written by a new writer who was overusing the same stylistic choices or AI. It made me sad.

My guess would be that the quality of AI writing would be narrower. Like AI isn't going to write something as evocative as Ursula K Le Guin, but it also is now good enough to not sound like someone's teenage fanfic where they accidentally changed a character's name in the middle ...

Anecdotally, it seems the newer models are noticeably getting worse at generating words for humans - as they are being more tuned for “agentic” work and code. (And probably training on more AI output )

AI stories are “created” from scraped human writings. So this is no surprise. When AI checkers first came out, I ran 20 pre AI client articles I’d written through them. 18 came back as “likely AI.”