Post

AI detectors rightly got a bad rap because they got it wrong often enough to be worse than useless — actively harmful. Pangram has a different algorithm that by all accounts simply works much better. I'd still be wary of basing an accusation on Pangram alone, but... www.nytimes.com/2026/07/29/b...

The case for using Pangram as at least an indicator of likely AI writing is much stronger than it was for Turnitin/GPTZero. Its widespread use would also deter a lot of folks from even trying to pass off AI slop as authentic. Is that enough to justify it, or is any nonzero error rate intolerable?

It might sound like a rhetorical question but I'm genuinely interested in how folks answer it. Obviously purveyors of AI writing don't want this. But what about people who hate AI slop but also disapprove of AI use in general, or worry deeply about false accusations? How do you weigh the tradeoff?

This is a great question. I'd love to see research on Pangram's accuracy for non-native speakers specifically. Logically its error rate couldn't be *that* high or else its overall error rate couldn't be so miniscule. But any amount of bias makes the tool's use problematic from a fairness standpoint.

This post is unavailable.

13/8/26, 1:38 pm2 reposts38 likes

Ben

@benfields.net

Well, it could be if there just isn't much (or any) non-native English in the test sets...

theatlantic.comAmerica Has a Pangram ProblemAI-detection tools are getting better. But they still aren’t good enough.

My colleague @matteowong.bsky.social argued persuasively in May that Pangram's accuracy actually makes it more dangerous, because it will tempt folks to overrely on it: www.theatlantic.com/technology/2... Still, its accuracy has improved further since then. Is there a threshold of acceptability?

I have not found it to be shockingly high but it definitely misses some AI writing. I will say I find it reassuring overall that they intentionally tuned it to err on the side of false negatives over false positives, which are far more harmful.

Jason Wilson@jasonaw.bsky.social

I can only counter this with my own anecdotes but I have found Pangram delivers a shockingly high proportion of false negatives

That said, even false negatives can cause problems, especially if there are non-negligible systematic biases. To the extent that people rely heavily on Pangram to flag potential AI writing, it would mean some folks will be a bit more likely to get away with using AI than others.

A surprising thing a Pangram engineer told me is that, at least so far, AI writing getting better over time has *not* made it any harder for Pangram to accurately detect. One theory is that making AI writing more humanlike is not the big labs' top priority right now; they're more focused on agents.

This post is unavailable.

Richard Lawler

@rjcc.bsky.social

I think my question is, unless you're grading essays, which maybe isn't the best approach now, do you really need an ai detector to tell the difference? You don't have to count em dashes or whatever, it tends to be obvious

Colin

@colin-fraser.net