NewsAnalysis
Inference prices keep falling. Here is who actually benefits
Per-token prices for frontier-class models have dropped by roughly an order of magnitude every 18 months. The savings are real, but they land unevenly across the stack.
Every few months a vendor announces that a model “matching last year’s frontier” now costs a tenth of what it did. The announcements have become routine enough that it is worth asking a boring question: who actually keeps the money?
The short answer is that the savings are real but they are not evenly distributed. A price cut at the model layer flows through to product margins only when a product’s cost structure is token-bound, its volume is fixed and its quality bar is already met. That describes fewer businesses than the headlines imply.
The trend, and why it holds
Tracking published list prices for a fixed capability level, the cost of frontier-class inference has been falling at roughly an order of magnitude every 18 months. Epoch AI’s price tracking and a16z’s “LLMflation” analysis arrive at similar slopes using different methods. Three mechanisms drive it:
- Distillation and small-model progress. Each generation of large models is used to train smaller successors that hit the previous quality level at a fraction of the parameters. The “GPT-4-class” bar has been cleared by progressively smaller models every year.
- Serving efficiency. Speculative decoding, better batching, quantisation and purpose-built inference chips all raise tokens-per-dollar without touching the model.
- Competition. With several labs shipping comparable capability, list price has become a lever. Open-weight releases anchor the floor, because anyone can serve them at cost.
None of those forces show signs of exhausting, so the sensible planning assumption is that a given capability will cost a third of today’s price a year from now.
Where the cuts land first
Cuts do not arrive uniformly across a vendor’s line-up. The pattern is consistent enough to plan around:
| Tier | Typical behaviour | What to expect |
|---|---|---|
| Small / fast | Cut aggressively and often | Near-zero marginal cost; used as a loss leader |
| Mid-size | Replaced by a better model at the same price | Quality improves, price holds |
| Flagship | Held at launch price, then superseded | Rarely discounted; deprecated instead |
| Long context and cached input | Fastest-moving line item today | Expect the largest percentage cuts here |
The last row matters most for builders. Output-token prices grab headlines, but for retrieval-heavy and agentic workloads the majority of spend is input tokens, and cached-input pricing is where vendors are now competing hardest.
Figure 1: A price cut rarely arrives evenly across a line-up. Cached and long-context input is where the steepest cuts are now concentrated.
Who benefits
Application builders with fixed workloads capture the savings directly. A support-ticket classifier that handles a set volume of tickets a day sees its bill fall with each cut. These businesses have strong incentive to re-evaluate their model choice every quarter, because yesterday’s premium model is now a mid-tier price.
Agentic products often do not. When a model gets cheaper, teams give agents more steps, longer memories and more retries, and the quality improves while the bill stays flat. That is a rational trade, but it means “inference is getting cheaper” does not translate to “our margins are improving”. Treat cheaper tokens as a quality budget, not a cost saving, unless you deliberately cap usage.
End users benefit through free tiers and higher limits more than through lower prices. Consumer subscription prices have been remarkably sticky even as the underlying cost fell; vendors pocket the difference or reinvest it in capability.
Cloud providers and inference startups are squeezed. Serving open-weight models is a commodity business with thin margins, and every list-price cut from a lab compresses them further. Expect consolidation.
Figure 2: The same price cut produces three different outcomes depending on how the product spends tokens.
What to do about it
- Re-benchmark quarterly. Keep an evaluation set for each production use case and re-run it against the current cheap tier. Downgrading a model that still passes your bar is the single easiest saving available.
- Budget in dollars per task, not dollars per token. Token prices are falling; tokens per task are rising. The product metric that matters is the former.
- Design for caching. Stable system prompts, shared context prefixes and deterministic tool schemas all increase cache hit rates, and cached input is where the steepest discounts are.
- Do not lock in. Multi-year committed-spend deals priced on today’s rates are a bet against the clearest trend in the industry.
Prices will keep falling. Whether that helps your business depends less on the vendors than on whether your product is built to let it.
Sources
- Epoch AI, "LLM inference price trends" Epoch AI
- a16z, "LLMflation: LLM inference cost is going down fast" Andreessen Horowitz
Frequently asked questions
Why are LLM inference prices falling so fast?
Three forces compound. Labs distil large models into smaller ones that match the old quality at a fraction of the compute; inference hardware and serving software extract more tokens per dollar from each GPU; and competition between vendors pushes list prices toward cost.
Do falling prices mean AI products get cheaper to run?
Only if the product's costs are dominated by tokens at a fixed volume. Many agentic products spend the savings on more steps, longer contexts and more retries, so the bill stays flat or grows.
