Post

LLM APIs need to return cost information in their response alongside tokens literally everyone is using models[dot]dev data to approximate this - we see so many reqs to its api but this is just sticker pricing, won't reflect discounts, etc so it doens't really work

Lawrence Jones

@lawrencejones.dev

What’s your recommendation here? Like you say the costs change in various ways, we have a registry.yaml file that tracks our current pricing and we write that into our database whenever we make a request so we know at that time what we spent. Not sure how you can do this nicely in the APIs.

16/5/26, 10:23 am0 reposts1 like

Lawrence Jones

@lawrencejones.dev

16 May

The layering gets pretty complex with the pricing you apply and can sometimes even be retrospective (if you negotiate a new price it can apply backwards) so quite tricky.