You open the pricing page for GPT-4o or Claude or Gemini and you see a number. A few dollars per million tokens, maybe less. You do some quick math in your head, it seems reasonable, you move on.
Three months later you are staring at a bill that does not match that math and you cannot figure out why.
Welcome to AI pricing. Where the number on the page is technically accurate and practically useless for predicting what you will actually spend.
This is not an accident. The way AI pricing is presented is not designed to help you estimate your costs. It is designed to look competitive against other providers at a glance. Those are very different goals and the gap between them is where your budget disappears.
The first thing the pricing page does not tell you
Every major AI provider lists two prices: one for input tokens and one for output tokens. Sometimes they are presented together, sometimes separately, but the distinction is always there if you look.
What they do not tell you is what your actual input to output ratio will be in practice, and how dramatically that ratio affects your real cost per useful unit of work.
Here is why this matters. If you are building a tool where users send short questions and the model writes long detailed answers, your output tokens are going to dominate your costs. You might be sending 300 input tokens and getting back 800 output tokens. Since output tokens are priced higher, often two to three times higher, the output side of your bill will be four to eight times larger than the input side.
Now imagine you are comparing two models. Model A charges two dollars per million input tokens and eight dollars per million output tokens. Model B charges three dollars per million input tokens and six dollars per million output tokens. Which is cheaper?
The answer depends entirely on your input to output ratio. If you are heavy on output, Model B is cheaper despite having higher input pricing. If you are heavy on input, Model A wins. The pricing page will never show you this because they do not know your use case. But you do, and you need to run the math with your actual ratio before you decide anything.
The caching variable that changes everything
Most providers now offer some form of prompt caching, where tokens that appear at the beginning of your prompt and stay the same across requests are charged at a reduced rate after the first time they are processed.
Anthropic, OpenAI, and Google all have versions of this. The discounts are significant, often fifty to ninety percent off the standard input price for cached tokens.
This means that the effective cost per token for applications with long system prompts or repeated context blocks can be dramatically lower than the headline price suggests. As SemiAnalysis noted in their deep dive on AI value capture and token economics, they estimated the true blended price for running certain agentic workloads at a fraction of the sticker price specifically because of cache hit rates above ninety percent on the repeated portions of each prompt.
If you have a two thousand token system prompt and you are sending a hundred thousand requests a day, and you are not using caching, you are potentially paying full price for two hundred million tokens per day that you could be paying a fraction of the price for. The pricing page lists caching as a feature. It does not tell you what ignoring it is costing you.
The tokenizer efficiency problem nobody mentions
Different models do not tokenize text the same way. The same paragraph of text will produce different token counts in GPT-4o versus Claude versus Gemini. For plain English the differences are small. For code, non-English languages, structured data, or text with lots of special characters, the differences can be significant.
This means that a model with a lower price per token can still cost more per request if it tokenizes your specific content less efficiently. You are not comparing price per token in the abstract. You are comparing price per token times tokens per unit of content, and the second number varies by model.
Nathan Lambert covered this dynamic in his analysis of GPT-4 pricing evolution at interconnects.ai, noting that the ten times reduction in price from GPT-4 Turbo to GPT-4o looked more modest in practice for certain content types because the tokenizer efficiency did not improve at the same rate as the headline price.
The only way to know your real cost per request on a given model is to measure it with your actual content, not estimate it from the pricing page alone.
Volume tiers that are almost never mentioned upfront
Most providers have pricing that changes at high volumes but the tier thresholds and the magnitude of the discounts are not always prominently advertised. You often have to contact sales or dig into documentation to find out what the pricing looks like at a hundred million tokens per day versus a million tokens per day.
If you are planning to scale, the pricing you see on the public page may not be the pricing that actually applies to your use case at that scale. Getting a clear picture of what the pricing structure looks like across different volume levels before you commit to an architecture or a provider is worth the time it takes.
How to actually compare models on cost
Stop looking at price per million tokens as your primary comparison metric. It is a starting point, not an answer.
The comparison you need to run is: take your actual system prompt, take a realistic sample of your most common request types including a range of input lengths, run them through both models you are comparing, measure the token counts for both input and output on each model, apply the full pricing including any caching discounts you would actually use, and calculate the cost per request for each model on your actual content.
That takes about an hour if you use the right tools. The Tokens per Dollar Calculator on Prompt Toolbox makes the pricing math fast so most of that hour is spent gathering your realistic test cases rather than doing arithmetic.
The model that comes out cheaper in that comparison is the cheaper model for your use case. It might be the same one that looked cheaper on the pricing page. It might not be. Either way, you will actually know rather than assuming.
The uncomfortable implication
The AI providers are not lying to you on their pricing pages. Everything listed there is accurate. But presenting price per million tokens without showing you what your actual ratio of input to output will be, without highlighting the impact of caching, without noting that tokenizer efficiency varies, and without showing you volume tier pricing, leaves you with a number that is accurate and incomplete at the same time.
Do not let an incomplete number make a complete decision for you. Run the actual math on your actual content before you commit to anything.