Post

People have real qualms with AI detectors, and so do I, but the evidence is mounting that Pangram in particular very rarely gets it wrong (especially on longer chunks of text, but increasingly even on short ones). @brianx.bsky.social tested it and it didn't miss: www.nytimes.com/2026/08/13/t...

AI detectors rightly got a bad rap because they got it wrong often enough to be worse than useless — actively harmful. Pangram has a different algorithm that by all accounts simply works much better. I'd still be wary of basing an accusation on Pangram alone, but... www.nytimes.com/2026/07/29/b...

The case for using Pangram as at least an indicator of likely AI writing is much stronger than it was for Turnitin/GPTZero. Its widespread use would also deter a lot of folks from even trying to pass off AI slop as authentic. Is that enough to justify it, or is any nonzero error rate intolerable?

It might sound like a rhetorical question but I'm genuinely interested in how folks answer it. Obviously purveyors of AI writing don't want this. But what about people who hate AI slop but also disapprove of AI use in general, or worry deeply about false accusations? How do you weigh the tradeoff?

This is a great question. I'd love to see research on Pangram's accuracy for non-native speakers specifically. Logically its error rate couldn't be *that* high or else its overall error rate couldn't be so miniscule. But any amount of bias makes the tool's use problematic from a fairness standpoint.

My colleague @matteowong.bsky.social argued persuasively in May that Pangram's accuracy actually makes it more dangerous, because it will tempt folks to overrely on it: www.theatlantic.com/technology/2... Still, its accuracy has improved further since then. Is there a threshold of acceptability?

That said, even false negatives can cause problems, especially if there are non-negligible systematic biases. To the extent that people rely heavily on Pangram to flag potential AI writing, it would mean some folks will be a bit more likely to get away with using AI than others.

A surprising thing a Pangram engineer told me is that, at least so far, AI writing getting better over time has *not* made it any harder for Pangram to accurately detect. One theory is that making AI writing more humanlike is not the big labs' top priority right now; they're more focused on agents.

My impression is that, if anything, the big labs *want* AI writing to be detectable. It's broadly assumed that they integrate watermarking techniques into their models, and their models develop identifiable mannerisms that they could train out but deliberately don't.

Honestly I think if Pangram ever did falsely flag one of my articles as being something other than 100% human-written, I would probably become militantly opposed to it (even knowing how rare that is). I don't blame anyone who has experienced that for never trusting it, or anything like it, again.

there is of course, famously, a FPR/FNR trade-off