Post

Surprising: frontier models (Claude, ChatGPT, Deepseek V4) produce the most predictable text of any local AI model or human text I've ever tested. Not surprising: Finnegans Wake is off the charts, by far least predictable—Shannon & Lydia Liu proved right—and Hemingway the most predictable human.

Why would that be surprising? LLMs - AI slop - is best known for being drab and predictable and non-creative.

What's surprising is that the giant trillion-parameter models are even more predictable than the tiny ones I can run on my own PC. That, plus the fact that 'aligned' or fine-tuned models are more predictable than their base model, suggests AI is optimized for predictability (a feature of slop?).

During training they are tasked to predict the next token and then get modified to predict the next token even better than before training. It's literally their job to be as predictable as possible! The bigger models have more parameters to do that even better.

This is perhaps hidden by the fact that they are commonly used with a randomized seed for the pseudo-random numbers used in calculation. Otherwise, the same input would always result in the same output and predictability would be off the charts. ;)

But they are not trained to predict Joyce, but a relatively large subset of all human speech in general. (And I think they are also trained on their own output, making them more self-consistent and self-predictable.)