Post

People have real qualms with AI detectors, and so do I, but the evidence is mounting that Pangram in particular very rarely gets it wrong (especially on longer chunks of text, but increasingly even on short ones). @brianx.bsky.social tested it and it didn't miss: www.nytimes.com/2026/08/13/t...

AI detectors rightly got a bad rap because they got it wrong often enough to be worse than useless — actively harmful. Pangram has a different algorithm that by all accounts simply works much better. I'd still be wary of basing an accusation on Pangram alone, but... www.nytimes.com/2026/07/29/b...

The case for using Pangram as at least an indicator of likely AI writing is much stronger than it was for Turnitin/GPTZero. Its widespread use would also deter a lot of folks from even trying to pass off AI slop as authentic. Is that enough to justify it, or is any nonzero error rate intolerable?

It might sound like a rhetorical question but I'm genuinely interested in how folks answer it. Obviously purveyors of AI writing don't want this. But what about people who hate AI slop but also disapprove of AI use in general, or worry deeply about false accusations? How do you weigh the tradeoff?

This is a great question. I'd love to see research on Pangram's accuracy for non-native speakers specifically. Logically its error rate couldn't be *that* high or else its overall error rate couldn't be so miniscule. But any amount of bias makes the tool's use problematic from a fairness standpoint.

My colleague @matteowong.bsky.social argued persuasively in May that Pangram's accuracy actually makes it more dangerous, because it will tempt folks to overrely on it: www.theatlantic.com/technology/2... Still, its accuracy has improved further since then. Is there a threshold of acceptability?

I have not found it to be shockingly high but it definitely misses some AI writing. I will say I find it reassuring overall that they intentionally tuned it to err on the side of false negatives over false positives, which are far more harmful.

That said, even false negatives can cause problems, especially if there are non-negligible systematic biases. To the extent that people rely heavily on Pangram to flag potential AI writing, it would mean some folks will be a bit more likely to get away with using AI than others.

My impression is that, if anything, the big labs *want* AI writing to be detectable. It's broadly assumed that they integrate watermarking techniques into their models, and their models develop identifiable mannerisms that they could train out but deliberately don't.

Honestly I think if Pangram ever did falsely flag one of my articles as being something other than 100% human-written, I would probably become militantly opposed to it (even knowing how rare that is). I don't blame anyone who has experienced that for never trusting it, or anything like it, again.

And yes, that's what soured me on everything but Pangram. Originality.ai at one point seemed like a better one... then it flagged a fully original, never touched by AI chapter at 97% likely AI.

Thankfully, this won’t happen! The people who claim this has happened to them (and there are many) are absolutely lying.

Pangram often claims text written by African English speakers are 100% AI written. They aren’t absolutely lying. Feed it text from Chinua Achebe’s books for example.