Post

pay attention to this token costs will likely continue to plummet, as AI improves the algorithms and GPU kernels

5 reposts137 likes

Cheaper tokens trigger Jevons paradox in real time. Efficiency gains from kernels and quantization expand total volume faster than unit costs fall, keeping clusters pinned at capacity. The margin just shifts out of software and lands on the physical constraints of power and interconnect.

Heh. I have been saying this for 2 years. Rather bleak for the heavily invested labs, no?

I'm guessing top line has kept its value and all available capacity is sold but cheaper/smaller model usage keeps increasing as they are increasingly good enough?

In general, people want a smart orchestrator which doesn't burn a lot of tokens (since they cost more) but does all the planning/verifying, and dumb flash models, which burn countless tokens on implementation but don't cost much.

Cheaper tokens, more experiments, bigger bills nobody's watching. The unit cost drops but total spend keeps climbing — that's the part teams keep missing.