Post

The only way for the cost of tokens to drop is if the models become orders of magnitude more efficient. This is very likely to happen, the only question being whether the AI bubble pops first. One way to drop the cost of the model is by employing higher reasoning, abstraction, etc.

IBM proved it was possible to beat a grandmaster at chess using a pure brute force approach in the 1990s. Deep Blue had no real concept of strategy. Modern chess programs that apply computation + strategy are much more powerful.

The $200/month Claude subscription I have is okish. Current frontier model capabilities are mediocre at best, but I can find $200/month worth of value here and there, so it's worth it for me. But, via the API though, I can burn $200 worth of tokens with a single prompt. 2/6

Most of the die-hard AI fans I talk to are using stuff like Cursor, Claude Code, ChatGPT, etc. All of those are heavily-subsidized to the point where those $20/month and $200/month subscriptions they're paying for are letting them burn thousands of dollars worth of tokens. 3/6

I'd guess some of the heavier subscription users are probably burning like $15-30k/month worth of resources. Then you have to consider the fact that the actual cost of API tokens may also be subsidized. These company are getting discount compute from cloud providers looking to expand. 4/6

it’s the Uber model, get everyone dependent on the cheap service as it’s gradually hollowed out and gets more expensive because of the factors it was subsidized to paper over

Except, instead of $3 rides across town, they just cook everyone's brain until they can't even remember how to drive a car

It’s insane and one day the bottom will fall out, and we will all be stuck either paying through the nose or looking dumb 😆 For fun go talk to the “Mac Mini gives me Opus output” guys to see how bad the brain rot is in the self-hosted model world. I guarantee you won’t go back there. 🤭

I've already had the misfortune of being surrounded with those people. They swear up and down that their shitty open weight model running on their $3000 desktop potato is on par with frontier capabilities

There's an entire industry of model and agent optimization that hasn't been explored yet. Besides the fact that model distillation is packing more capabilities into smaller models (e.g. tiny on-device models bring as good as the largest previous generation), there's routing, text diffusion, caching

This was addressed in post 6 of the 6 post thread where I talk about "handwavy napkin math that makes AI becomes basically free"

Yes, post 6 is just as handwavy though. There's real evidence of breakthroughs with Deepseek, Gemini Nano, etc. To claim that there's nothing left to be done in optimization isn't very convincing. The companies who focus on innovating in that area will outcompete the others.