Post

As a former academic I take some issue with this, since just having the tool do work for you is not learning. It's going to a restaurant and pretending you're a chef. If you use the tool to help understand what's going wrong with what you made, i.e. as a tutor, that's a lot better

So the situation I was really talking about is one where you have to learn a lot of technical skills before you can get to the thing you want to actually learn.

In particular, if you want to make videogames, you generally have to learn a lot about programming before you can really get to the part where you're designing games. My hope is that LLMs will facilitate people being able to get to a place where they can design a game, test it, and iterate on it...

...more quickly, so that they don't have to learn a bunch of learning programming to even get to the place where they are developing the skills the care much more about and potentially end up becoming discouraged because of it.

I also suspect "well this game I designed that an LLM wrote most of the code for has a bug in it and the LLM keeps screwing up and I want this thing I care about to be better" might be a motivator for learning those technical parts?

Like, I don't actually know if this happens. The only real way we'll find out is if a game designer 20 years from now talks about how they got their start vibecoding their own kinda-crappy games and then learning to code. But I'd like to think there's a chance?

I'd take exception to a lot of those technical skills being useless, unless the goal is not to understand the code you're producing. But really, the biggest blocker here is wealth inequality. The cost to do the kind of work you're talking about is nontrivial now and will only get worse.

Compared to the cost of producing code the trad way the cost is quite trivial and trending down.

"trending down" Citation desperately needed. This does not fit the world.

The cost of using an LLM to solve problems has been dropping over time. Having an LLM help debug your code is certainly cheaper than the rates I'd charge you for the same.

Just judging by the amount the big guys charge for their subscriptions, I don't see it. They're not getting cheaper. At least for the moment they don't seem to be getting more expensive, but I wouldn't expect that to hold unless it gets a lot cheaper and easier to build data centers

The price of inference has dropped sharply in even just the past six months, and open-weight models are putting pressure on OAI/Ant to keep costs down. I don't expect that to continue indefinitely, but inference is indisputably getting cheaper right now.

And no, that is not just because the providers are selling at a loss. Snaller models are getting better. The capabilities of local models have expanded dramatically.

I do not have problems with people disliking AI. I think disliking AI is very reasonable. I am, however, starting to become exasperated with people making claims about AI that are BLATANTLY false and can be proven to be so with a five-minute Google search.

So why isn't Claude cheaper? No, really, I'm serious. The prices are at best static.

Claude IS cheaper. A lot cheaper, actually. What you are looking at is the price of token plans. A token plan is a subscription that allows you a certain amount of usage of the LLMs a provider hosts per month. The rate at which your usage is consumed varies depending on what model you use.

How does usage map to actual amounts of tokens for different models? Well, the provider I pay makes it fairly clear, but the labs... don't. Which isn't great! But generally, smaller models consume less usage because they are cheaper to run.

TOKEN PLANS ARE NOT THE PRIMARY METHOD BY WHICH INFERENCE IS SOLD. My understanding is that the labs make money on token plans, but the real goal of token plans is to get engineers using the tool ls so that they want them at work. And that's where the REAL money gets made.

Businesses (or at least large ones) do not get to buy token plans. They have to pay per-token rates. LLM tokens are priced in $/MTok. That is, dollars per million tokens. How text becomes tokens are and how big they are varies, but apparently they are approximately four characters on average.

Token pricing is somewhat confusing. There are three prices associated with a model: cost per input token, cost per output token, and an extra-low price for sending input tokens you've sent recently. All of these prices have been going down for a given level of model capability.

This is where people generally say "AHA! But some newer tokenizers turn the same text into more tokens! And models tend to EMIT more tokens! So it's not actually cheaper!" The former two statements are true: token price alone does not quantify the price difference between using two different models.

However, *even if you account for that*, newer models are still cheaper. The price of completing actual benchmark tasks with newer models is lower, and this seems to hold for Real Work as well.

My understanding is that this has been reflected in usage limits. That is, your $20/month on a token plan gets you more inference than it used to because the models are more efficient. This IS true for the provider I pay, but I don't pay OAI or Ant, so.