Post

The reason why procgen in games is, as we like to say, a great way of getting twice the content with four times the work is that you can't fight Shannon's laws any more than you can fight the laws of thermodynamics. You can't give the audience more information than you were willing to put in.

It's a magic trick of the same kind as the ones where something large comes out of a small box; it's done by making something that *unfolds* itself. "Generation" is almost a misnomer, really, it's *decompression.*

Any aspect of the system that isn't downstream from those choices is, essentially, noise. A little bit of noise is texture, a lot of noise makes the semantic content of the piece collapse out.

This is the trap that LLMs and other generative AI systems present in this context; if you're generating vast amounts of content from a tiny prompt, what you're doing is supplying your audience with almost nothing and filling in the empty space with the semantic equivalent of sawdust.