Why AI agents cost more than you modelled — and what to do about it

Teams model agent costs by taking the price of a chat turn and multiplying by the number of steps they expect. The result is consistently, sometimes dramatically, too low. Not because the estimate of steps was wrong — although it usually is — but because an agent step is structurally more expensive than a chat turn, for reasons that compound.

There are four of them. Each is fixable, and none is fixed by choosing a cheaper model.

1. Every step re-establishes the whole context

A chat turn sends the conversation and gets an answer. An agent step sends the goal, the tool definitions, the accumulated scratchpad of everything done so far, the results of previous tool calls, and the operating instructions — and then does it again for the next step, with the scratchpad now longer.

The input cost of a ten-step agent run is therefore not ten times a single step. It is closer to the sum of a growing series, because step ten carries everything steps one through nine produced. Tool results are the aggravating factor: an agent that reads three documents is carrying those documents in its context for every subsequent step of the run.

2. Tool definitions are billed on every call

A well-equipped agent might have twenty tools, each with a name, a description and a JSON schema for its parameters. That block can run to thousands of tokens, and it is sent with every single step of every single run, whether or not the agent uses any of those tools.

• Trim descriptions to what changes the model's behaviour. A schema is documentation for the model, not for a human reading the codebase.
• Scope the toolset to the task. An agent handling a billing question does not need the deployment tools in its context.
• Consider staged toolsets: a small router picks a capability area, and only that area's tools are loaded for the run.
• Keep the tool block at a stable position in the prompt so it is cached rather than re-billed.

3. Loops are unbounded unless you bound them

The defining feature of an agent is that it decides how many steps to take. That is the capability you wanted and it is also an open-ended commitment to spend. An agent that misreads a tool error can retry the same action twenty times, each attempt costing more than the last because the scratchpad keeps growing.

Most catastrophic AI bills we have seen were loops, not model choice. The controls are simple and belong in the code rather than in a policy document:

1. 1A hard maximum step count per run, enforced by the runtime and not by the prompt. The model is not a reliable enforcer of its own limits.
2. 2A per-run token or cost budget, checked between steps, that terminates the run and reports partial progress rather than continuing.
3. 3Repetition detection: if the last three steps are substantially the same action, stop. Agents get stuck in ways that are trivial to detect externally and invisible from inside the loop.
4. 4A per-tenant or per-key spend cap in the gateway, so a pathological run cannot consume the estate's budget.

4. Failures are silent and expensive

When a chat answer is wrong, a user sees it. When an agent's third step is wrong, the agent keeps going, builds on the error, and often arrives at a result that looks plausible after twelve steps that should have been four. All of it is billed, and none of it appears as a failure in any provider dashboard — every one of those calls succeeded.

This is where cost per successful task earns its keep. A run that completes the customer's task in four steps and a run that flails through twelve before producing the same answer look identical to per-token monitoring and completely different on the ratio that matters.

What good agent cost control looks like

Step-level tracing deserves a note. Agent runs are the workload where per-call telemetry stops being a nice-to-have: without a task ID that ties every step of a run together, you cannot tell a cheap run from an expensive one, and the average tells you nothing because the distribution has a long tail that contains most of the money.

Model choice comes last here too

The instinct with an expensive agent is to move it to a cheaper model. It is usually the wrong first move — and occasionally the wrong move entirely, because a weaker model takes more steps to reach the same place, and step count multiplies every other cost term in the run.

A more productive decomposition is to route within the agent rather than around it. The planning and synthesis steps, where reasoning quality determines how many steps the run takes, keep the strong model. The mechanical steps — parsing a tool result, deciding whether output satisfies a schema, extracting a field — go to a small model. That is routing by task class applied inside a single run, and it captures most of the saving without lengthening the run.