What an AI agent costs to run once it is in production

The price of a token is the wrong number to watch

The question a CTO asks most often about an AI agent is what it will cost to run. The number every provider publishes, the price per token, explains less of that bill each year. Token prices are falling fast. Stanford's 2025 AI Index found that the inference cost of a system performing at the level of GPT-3.5 dropped more than 280-fold between November 2022 and October 2024, and Epoch AI measured price declines of between 9x and 900x a year, depending on the task.

Bills are rising anyway. In August 2026 Gartner predicted that AI inference costs per agentic workflow will increase more than fivefold through 2028, noting that routing a task to an agentic reasoning model costs at least five times as much as a basic chatbot interaction. In McKinsey's 2026 State of AI survey, about 20 percent of respondents said AI operating costs, token costs included, constrained their use of AI. The unit got cheaper. The work done per task got much larger.

What actually makes up the monthly bill

An agent, of the kind our AI engineering practice builds, does not make one model call. It plans, retrieves, calls a tool, reads the result and calls the model again, often several times before a task is done, and every call carries the instructions, the retrieved documents and the work so far. The useful unit is therefore the completed task, not the token: model calls per task, times the tokens each call carries, times the price of the model tier, times the number of tasks, plus the retries and failed attempts no user ever sees.

Then come the costs that never appear on a provider invoice. The retrieval index and the pipelines that keep it current. The tracing that records what the agent did and why. The evaluation runs that test every change. The people who review the cases the agent escalates. Data residency adds to the token price itself: as of October 2026, OpenAI's pricing page charges a 10 percent uplift on regional data-residency endpoints for models released on or after March 5, 2026, and Anthropic's pricing page applies a 1.1x multiplier to US-only inference on Claude 4.6 and later models.

When Caltor scopes an agent, the cost model is written per task before the architecture is fixed, because that is the number a finance team can hold against the value of the work. Spend is then attributed to the workload that caused it in the observability we build into the data and platform layer, so the estimate is checked against the real bill from the first week.

The model underneath you has a retirement date

Every major provider retires models on a published schedule. OpenAI commits to at least six months' notice before retiring a generally available model, and as little as two weeks for a preview. Anthropic commits to at least 60 days' notice for publicly released models; claude-sonnet-4-20250514 was deprecated on April 14, 2026 and retired on June 15, 2026. Google keeps a Gemini model on Vertex AI available for at least 12 months after release; gemini-2.0-flash, released on February 5, 2025, was retired on June 1, 2026. And on Amazon Bedrock the move is yours to make: its documentation says plainly that migration will not happen automatically.

That Gemini model lasted just under 16 months from release to retirement. A system expected to run for three years should budget for more than one migration for every model it depends on.

A migration is not a configuration change. A new model behaves differently on your prompts, so the evaluation suite has to run, the failures have to be read and the prompts adjusted before the switch. It is not cost-neutral either. Anthropic's pricing page notes that the newer tokenizer in Claude 4.7 and later models produces approximately 30 percent more tokens for the same text, so the same workload can cost more on a model with a lower list price. The evaluation suite is the contract that decides whether the new model ships, and it is also where the new cost per task gets measured.

Upkeep is a line in the budget, not an accident

Machine learning systems have always cost more to keep than their builders expected. In 2015 a team of Google engineers wrote that it is common to incur massive ongoing maintenance costs in real-world ML systems, and that only a small fraction of such a system is the ML code itself (Sculley and others, NeurIPS 2015). The NIST AI Risk Management Framework says the same of today's systems: they may require more frequent maintenance because of data, model or concept drift.

In practice the upkeep has four parts: model and prompt changes tested against the evaluation suite, new evaluation cases for the failures real use uncovers, monitoring of quality, latency and spend, and the engineering time to fix what breaks. It is the work our managed operations engagement exists to do, and it belongs in the budget whether Caltor does it or your own team does.

Leaving it out is how projects get canceled. In June 2025 Gartner predicted that over 40 percent of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls.

Where the cost comes down

Most of the saving is design, not negotiation. Send each step to the cheapest model that passes the evaluation suite for that step, and keep the expensive reasoning model for the steps that need it. Do deterministic work, such as a lookup, a calculation or a format check, in ordinary code rather than in a model call. Keep the context short: an agent that carries its whole history into every call pays for that history every time.

Use the pricing mechanics the providers publish. On Anthropic's pricing page, reading a cached prompt prefix costs a tenth of the normal input price on most models, and work sent through the Batch API is billed at half price. Both reward a stable system prompt and a queue for anything that does not need an answer within seconds.

McKinsey's 2026 survey describes the organizations getting value from AI as treating operating costs as a design constraint, not an afterthought. That is the whole discipline. The cost per task is decided in the architecture, and the architecture is far cheaper to change than the bill.

Budget it per task, before you build

Before funding an agent, ask for four numbers: the model calls and tokens a completed task takes, what one task costs at today's prices, how many tasks run in a month, and the upkeep for a year, including at least one model migration. Then hold the cost of a task against what the task is worth when a person does it. If the agent only pays off at the most optimistic volume, the project is fragile before it starts.

Usage-based pricing can also change the build versus buy decision. A vendor that charges per action can be cheap at pilot volume and expensive at full volume, and a system you run yourself puts the cost per task under your control. Our AI strategy work ends in exactly this kind of costed plan, written by the engineers who would build it. If you have an agent on the roadmap and no number for what it costs to run, book a conversation with a founding partner.