A single language-model call is easy to budget: input tokens times the input price, output tokens times the output price, done. Agents break that intuition. The moment a model starts calling tools, reading results, and deciding what to do next across multiple steps, cost stops scaling with the number of steps and starts scaling with something closer to their square.
If your first agent prototype ran fine in testing and then produced an alarming invoice at any real volume, this is almost always why. Here is the mental model that explains it and the levers that fix it.
Why agent cost compounds
Each step of an agent is a fresh model call. And each call has to re-send the accumulated conversation so far — the system prompt, every prior tool call, every tool result, and all the reasoning in between — as input. The model is stateless; the only way it "remembers" step three when doing step four is that you pay to send steps one through three again.
So a ten-step task is not ten times the cost of a single call. The early steps are cheap; the later steps carry the full history of everything before them. Total input tokens across a task grow with roughly the square of the step count, not linearly.
total input tokens per task ≈ tokens_per_step × steps × (steps + 1) / 2Double the steps and you roughly quadruple the input bill, even though output stays linear. You can put your own numbers against this with the AI agent cost estimator.
Input versus output, and why the split matters
Output tokens usually cost several times more per token than input tokens. But agents are input-heavy: a long tool result read once produces far more input tokens on every subsequent step than the short decision the model writes in response. In most agent workloads, the compounding input is the dominant cost, which is the opposite of a simple chat app where output often dominates.
That inversion changes what you optimize. Trimming a verbose model response saves a little; trimming what gets carried forward as context saves a lot, on every step that follows.
The three levers
1. Fewer steps
The cheapest token is the one you never send. Better prompts, better-scoped tools, and clearer stopping conditions all shorten tasks. Because cost is quadratic in steps, shaving two steps off a ten-step task saves far more than the raw step count suggests.
2. Prompt caching on repeated context
The system prompt and the earlier turns recur verbatim on every step — which is exactly what prompt caching is built for. Cached input is billed at a fraction of the normal rate. For agents with a long, stable system prompt, caching is frequently the difference between an economics that works and one that does not.
3. Context management
Do not carry the full transcript forever. Summarize or drop old tool outputs once they have served their purpose. An agent that compacts its history grows its per-step input slowly; one that appends every raw tool result grows it fast. Model choice per step helps too — not every step needs your most capable model.
Budget for the distribution, not the average
Real agents have variable step counts. Easy tasks finish in two steps; hard ones retry, loop, and explore. A single average understates cost when the distribution has a long tail, and failed tasks still burn tokens before they fail. Instrument your agent, log step counts and token usage across real tasks, and budget against the worst case as well as the mean.
The takeaway
Agents are not expensive chat calls; they are a different cost shape. Once you internalize that input compounds with step count, the optimization priorities fall out on their own: end tasks sooner, cache the repeated context, and stop carrying history you no longer need.