Post

Stargirl
@thea.codes

Every now and then I throw some rocks at the latest AI shit just to know my enemy. This "agentic" shit is wild. Less than useless with open weight models. Barely cobbled together harnesses. The amount of tokens, hardware, and energy it takes to puke out useless slop is astounding.

Stargirl
@thea.codes

Unless we just see entirely new hardware approaches with orders of magnitude levels of improvement, there's no way in hell the unsubsidized cost of this isn't fucking ridiculous.

0 reposts17 likes

emily s.
@emily.news

The margins on frontier model inference actually do seem to be positive, from what I've read. I imagine they benefit massively from economies of scale, specialized models, aggressive shared KV caches, and the like.

emily s.
@emily.news

IDK if that number is correct, or if it accounts for things other than inference, but there's a nice discussion here bsky.app/profile/simo...

Simon Willison@simonwillison.net

They're rumored to make a healthy margin on inference - so the more tokens they can sell the better Especially since corporate clients can't sign up for the $100 or $200/month subscription plans - companies have to pay the full API prices

Stargirl
@thea.codes

I'm just so unbelievably skeptical of any numbers these clowns put out. Even if inference is, in isolation, profitable, does it actually recoup the cost of training before becoming obsolete? It's also so funny that agentic shit burning through tokens is how they're selling more.

Stargirl
@thea.codes

This is also maybe the first time where I thought I might be holding it wrong, especially for the local LLM use case. Qwen3.8 + pi/omp was just... shit. I asked it to *plan* a simple refactor and it churned and churned and eventually got stuck in a loop where it didn't realize it finished lmao.

Stargirl
@thea.codes

nevermind the fact that the result was ass. I'd have gotten more utility out of running a GPU benchmark and letting my cat sit on my computer for the warmth.