Post

I was very bearish on AI to begin with, but recently I started experimenting more with APIs and building my own agents. And wow, I did not realize how heavily subsidized all the subscriptions were: 🧵 1/6

The $200/month Claude subscription I have is okish. Current frontier model capabilities are mediocre at best, but I can find $200/month worth of value here and there, so it's worth it for me. But, via the API though, I can burn $200 worth of tokens with a single prompt. 2/6

Most of the die-hard AI fans I talk to are using stuff like Cursor, Claude Code, ChatGPT, etc. All of those are heavily-subsidized to the point where those $20/month and $200/month subscriptions they're paying for are letting them burn thousands of dollars worth of tokens. 3/6

11 reposts75 likes

I'd guess some of the heavier subscription users are probably burning like $15-30k/month worth of resources. Then you have to consider the fact that the actual cost of API tokens may also be subsidized. These company are getting discount compute from cloud providers looking to expand. 4/6

Do you have recommended reading here? My understanding is that the subs are super subsidised indeed. But the actual API calls (which is what many enterprise customers are paying) don't immediately feel very subsidised to me. Are you saying AWS/GOOG are providing Anthropic with subsidised capacity?

The API token cost is *absolutely* subsidized by investors doing loss lead bets. But the more you squint, the dumber it gets. In talking with one of the largest players earlier this year, they noted that it's nearly impossible to calculate the true cost of serving tokens once subsidies stop.

Yeah, it's crazy. Every time I bring it up people are just like "well technological advancements will make it cheap". How? Like what is your basis for this? And then they just hand wave it away

Compare the cost of Microsoft’s Arm accelerator MAIA (coming up on gen 3) with general purpose inferencing infrastructure and you see why the compute cost of tokens is going down (but token usage is going up in probably a mirror image of the cost graph)

This argument (I refuse to call it "line of thinking" because it is anything but) drives me bananas. Nobody who understands the first thing about how this house of investment cards is built would make such a statement.

So, like, yeah, "everyone gets a free supercomputer" is great and all. But it isn't a serious business model. I just don't see how any amount of capability advancement or decrease in operational cost makes generative AI even remotely good value for money. 5/6

Then every AI proponent I talk to about this just come up with some insane napkin math where token costs keep decreasing exponentially forever. It feels as if the entire industry is basically operating on some insane Moore's law hypothesis where everything just becomes free... 6/6

AI is venture capital's last and most desperate attempt to get Moore's Law back after Dennard Scaling effectively killed its performance gains. It's a desperate attempt to undo dark silicon. And that happened 20 years ago so it's past necrophilia into necromancy

I think you might be just a hair too young to viscerally remember the dot com bubble bursting, but this is exactly what the run-up to that felt like, with people saying “hang on, how does pets dot com sell 40lb bags of cat food at invoice price with free shipping and expect to make a profit?”

Can't talk about the economics of the whole thing but using the free Claude tokens via the web interface (no API, no agents, etc.) let me do in two months what before I couldn't do in 5 years, so it has been quite useful to me. Open-source projects, though, so I don't have any cost/benefit analysis.

What’s wild to me is that OpenAI thinks they’re going to churn most of their $20/mo subscriptions into $8/mo subscriptions by the end of 2027. Do businesses usually plan to scale by moving most of their customers to lower-revenue users?

In my field it is faster, better and (so far) cheaper than anything else before. We just structure data with it and it seems to have solved many parts of NLP. Smaller models are enough for this and it produces ROI in BPA. So, even without being a "fan" we have to use it.

It's not sustainable, unless you keep stealing money from the public purse which is what these techbros are doing with their alliance with Trump. Even that model is ultimately not sustainable but they will bleed us and planet dry until they get there (or we get there first with the guillotines).

Can't it be the inverse, API is insanely overpriced because it's targeted at? Not defending AI companies or LLMs, fuck Sam Altman, I just didn't understand why you concluded that the API price is the fair one, and not the other way around?

Google did find a way to reduce memory usage by 6x. But the only way I see LLMs being affordable is local models that run on your own device.