If your AI running costs went up this year while every headline told you AI was getting cheaper, you are not being overcharged and you have not misread the pricing page. Both things are true at the same time. The price of a unit of model output is collapsing. The number of units required to do the work people now expect is rising faster. Measured together, the bill goes up.
This post is the follow-on to what AI actually costs a small business, which answered the question before you started. This one answers the question you get twelve months in, which is why the invoice does not match the story.
The number that is falling
Researchers at MIT FutureTech tracked model prices across three benchmarks covering knowledge and reasoning, mathematics and software engineering, using archived pricing snapshots from April 2024 to November 2025. Sample sizes differ by benchmark, from 138 price observations across 93 models on the largest down to 21 observations across 19 models on the smallest. On two of the three, for models sitting on the cost-performance frontier, they found prices "reducing annually by a factor of 5 to 10 times". Looking at open-weight models only and adjusting for falling hardware prices, the remaining decline is around threefold a year, which they read as the contribution of algorithmic progress.
Epoch AI reached a similar figure independently, describing the trend as "very roughly a 5-10x cost reduction per year for reaching a given capability level". Their worked example is concrete: reaching about 27% accuracy on the FrontierMath benchmark took roughly 43 million output tokens in April 2025 and roughly 5 million in December 2025.
So the story is not marketing. Per unit of capability, this is getting dramatically cheaper, faster than almost any other input your business buys.
The number that is rising
The same analysis found that the cost of evaluating frontier-level models on those benchmarks "increased at an approximately exponential rate, on the order of 3 to 18 times per year", because newer models think for longer and produce far more tokens per answer. The sharper finding sits on one benchmark, GPQA-Diamond, where the authors say the effect is particularly strong: "roughly half of measured progress is associated with increasing inference prices". Hold the cost fixed and performance improves at about half the rate the raw time trend suggests. That is a correlation drawn from their regression rather than a causal split, and the effect is weaker on the other benchmarks, so do not carry it further than they do.
That is the mechanism. You did not change your usage pattern. Your provider released a model that reasons before answering, your team switched to it because it is better, and the same question now consumes several times the tokens it used to. The per-token price went down. The per-answer price went up.
Epoch is careful about its own earlier data, noting that the older price-trend dataset "doesn't account for reasoning models or tasks where longer outputs provide genuine value" and is vulnerable to benchmark contamination. That caveat is the difference between a trend and a promise, so it belongs in the open rather than in a footnote.
Accuracy has a price, and it is larger than people expect
A study published in June 2026 compared retrieval-augmented generation against long-context prompting on a document-grounded task, across 972 expert-validated answers. Long-context prompting was more accurate: 73.1% correct against 65.4% for semantic retrieval. It also cost twenty-six times as much per query in tokens.
The study used small language models on a single manufacturing case study and is a preprint, so the exact multiple is indicative rather than a constant for your workload. The shape transfers. Buying the last several percentage points of accuracy by stuffing everything into the context window is not a small premium. It is an order of magnitude, and it recurs on every request forever.
That is a business decision, not an engineering one, and whoever owns the outcome should make it. Sometimes eight points of accuracy is worth twenty-six times the cost. Usually it is not, and the answer is better retrieval, which is the argument we made in start from the decision, not the data.
The rand problem nobody prices in
There is a cost line that does not appear in any of the research above, because the research is not written for us.
Microsoft prices its Copilot business plans to South African customers in United States dollars on its own South African storefront. As at 14 September 2026 the business add-on shows USD 18.00 per user per month on an annual commitment, promotionally reduced from USD 21.00, with the page stating "Price does not include VAT." Most AI tooling a South African business buys works the same way.
Two exposures follow, and only one of them is obvious. The rand closed at about R16.14 to the dollar on 11 September 2026, inside a twelve-month range of roughly R15.64 to R17.58 on Investing.com's figures. On our own arithmetic that eighteen-dollar seat runs from about R282 excluding VAT at the strong end of the band to about R317 at the weak end. The second exposure is the promotion: at that rate, reverting from eighteen dollars to twenty-one puts the same seat near R339. Neither movement is something you control, and both are larger than most of the savings you will find by tuning prompts.
Every rand figure here is our own arithmetic on a dollar price, using a rate with a date attached. Re-rate on the day before you put a number in a budget, and be sceptical of any rand price quoted to you for a twelve-month term without a stated rate and date.
For context on the scale of what is happening around us: Gartner figures reported by TechCentral put South African IT spending for 2026 at about USD 28.1 billion, growing 19.8%, with the data centre systems segment growing 90.6%. Gartner supplied that local cut to the publication on request and has not published it itself, so TechCentral is the citable source rather than a Gartner release. Its May 2026 revision put total worldwide AI spending for 2026 at about USD 2.59 trillion, of which AI infrastructure, a category that includes devices and semiconductors as well as servers, accounts for roughly USD 1.43 trillion. Our entire national IT spend is close to one percent of global AI spending alone, which is worth remembering the next time a vendor tells you that South African businesses are being left behind. We are a small market buying in someone else's currency, and that is a procurement fact rather than a moral one.
The four levers that actually move the bill
In roughly descending order of how much money they save per hour of effort.
- Cache the parts that do not change. Both major providers price cached input at roughly a tenth of fresh input. If your prompts carry a stable system prompt, a schema, a style guide or a document the user is asking repeated questions about, caching is the single largest lever available and it requires no change to output quality.
- Batch anything that is not interactive. Overnight classification, summarising yesterday's tickets, enriching a CRM export. Batch processing is typically half price, and nobody is waiting.
- Stop sending the frontier model easy work. Most production traffic is routing, extraction, classification and formatting. A small model does that at a fraction of the cost. Reserve the expensive model for the requests that actually need judgement, and measure the split, because it is usually worse than anyone guesses.
- Retrieve instead of stuffing. The twenty-six-times figure above is what you pay for not having done the retrieval work. Better retrieval is cheaper than a larger context window, permanently.
And one that is not a lever so much as a discipline: kill the workflows that were never worth automating. We set out a filter for that in AI workflows worth automating, and the cheapest token is still the one you do not spend.
What to measure, so this conversation is shorter next year
Track cost per completed task, not cost per token and not total spend. Total spend tells you nothing about whether the thing is working, and cost per token will fall while your bill rises, which is exactly the confusion this post exists to explain. Cost per completed task moves when the model changes, when the prompt changes, when usage changes, and when someone quietly switches to the expensive model, which is the failure mode nobody catches for a quarter.
Pair it with the honest denominator: how many of those tasks were actually correct. We have written about how easy it is to get this wrong in measuring AI ROI without lying to yourself, and the cost side is where the self-deception is cheapest, because a falling per-token price gives you a true statement to hide behind.
Gartner's January 2026 commentary placed AI in what it calls the trough of disillusionment, with enterprises prioritising proven outcomes over speculative potential. The spend figures above are from its later May revision, not that release. That is a reasonable place to be. If you are choosing what to prove first, that is the conversation we have in AI consulting, and it usually starts by finding the workflow where the cost per completed task is already defensible.
FAQ
Why did my AI bill go up when providers cut prices? Because newer models produce far more tokens per answer. The MIT FutureTech analysis found frontier prices falling five to ten times a year while the cost of running a frontier model on a hard task rose three to eighteen times a year. Per unit of capability you are paying less. Per answer you are paying more.
What is the single biggest saving available? Prompt caching. Both major providers price cached input at around a tenth of fresh input, and most production prompts contain a large stable portion. It costs engineering time once and saves on every request.
Should we use a cheaper model? For most production traffic, yes. Routing, classification, extraction and formatting do not need a frontier model. The useful exercise is to measure what share of your calls actually require judgement, then route accordingly, rather than defaulting the whole system to the best model available.
Is long context better than retrieval? It was measurably more accurate in one 2026 study, 73.1% against 65.4%, at twenty-six times the token cost per query. That was a single preprint using small models on one case study, so the multiple will differ for you. The principle holds: accuracy bought by context stuffing is expensive and recurring.
How should a South African business budget for this? In dollars, with a stated rate and date, and with a currency band. Most AI tooling is priced in USD even on South African storefronts, and the twelve-month rand range has recently been wide enough to swing a seat price by more than any prompt optimisation will.
What number should we report to the board? Cost per completed task, alongside the share of those tasks that were correct. Total spend and cost per token both mislead, in opposite directions.
References
- Gundlach, H., Lynch, J., Mertens, M. and Thompson, N., "The Price of Progress: Price Performance and the Future of AI", MIT FutureTech, arXiv:2511.23455v2, last revised 23 March 2026. Note that v1, dated 28 November 2025, carries a different title. https://arxiv.org/abs/2511.23455
- Denain, J.-S., "How persistent is the inference cost burden?", Epoch AI Gradient Updates, 16 February 2026. https://epoch.ai/gradient-updates/how-persistent-is-the-inference-cost-burden
- Epoch AI, "LLM inference prices have fallen rapidly but unequally across tasks", data insight, last updated 12 March 2025. https://epoch.ai/data-insights/llm-inference-price-trends
- Hamilton, J. et al., "The Token Tax of Epistemic Accuracy: Comparing RAG and Long-Context Architectures for Document-Grounded Generative AI Applications", arXiv:2606.20898, 18 June 2026. https://arxiv.org/abs/2606.20898
- Anthropic, Claude API pricing documentation (accessed September 2026). https://platform.claude.com/docs/en/about-claude/pricing
- OpenAI, API pricing (accessed September 2026). https://openai.com/api/pricing/
- Microsoft, "Microsoft 365 Copilot for business", South African storefront, accessed 14 September 2026, showing USD 18.00 per user per month on annual commitment, reduced from USD 21.00, excluding VAT. https://www.microsoft.com/en-za/microsoft-365/copilot/business
- Investing.com, USD/ZAR, close of 11 September 2026 and twelve-month range. https://www.investing.com/currencies/usd-zar
- TechCentral, "South African IT spending surging as AI boom lands locally", 29 July 2026, reporting Gartner figures supplied to the publication on request. No Gartner release carries the South African breakdown. https://techcentral.co.za/south-african-it-spending-surging-as-ai-boom-lands-locally/284295/
- Gartner, "Gartner forecasts worldwide AI spending to grow 47% in 2026", press release, 19 May 2026, revising total AI spending for 2026 to USD 2.59 trillion with AI infrastructure at USD 1.43 trillion. https://www.gartner.com/en/newsroom/press-releases/2026-05-19-gartner-forecasts-worldwide-ai-spending-to-grow-47-percent-in-2026
- Gartner, "Gartner says worldwide AI spending will total 2.5 trillion dollars in 2026", press release, 15 January 2026, the superseded 4Q25 vintage, cited here only for the trough of disillusionment positioning. https://www.gartner.com/en/newsroom/press-releases/2026-1-15-gartner-says-worldwide-ai-spending-will-total-2-point-5-trillion-dollars-in-2026
Rand conversions in this post are our own arithmetic on published dollar prices, not figures quoted by any vendor, using a rate dated in the body. Check the rate and the vendor's current price on the day before you quote either.
Written by JP Dippenaar, Sixees Labs. Last reviewed September 2026.