How to Control AI Agent Token Spend

If you are building agentic workflows on top of LLM APIs, you have likely seen the billing surprise: a multi-step reasoning chain ran longer than expected, an error-retry loop had no exit condition, or a sub-agent spawned sub-agents and the input context grew quadratically with each iteration. Token spend on agentic systems is structurally different from single-prompt usage — and the patterns that keep costs bounded are not obvious until you have burned through a budget ceiling at least once.

This article covers four concrete mechanisms: understanding why loops accumulate tokens faster than expected; recognising the three runaway loop patterns before they complete; setting hard caps and budget gates in your agent code; and applying model routing to match task difficulty to model cost tier. Code examples below are in Python and work with any LLM SDK that exposes token usage in the response object.

Why agent loops burn tokens

Every API call bills for input tokens and output tokens separately. On agentic workloads, input token accumulation is the primary cost driver — not output. Here is why.

Each iteration of an agent loop passes the full conversation history as new input to the next call. If an agent emits 800 tokens of output on iteration 1, those 800 tokens become part of the input for iteration 2. On iteration 2 the agent emits another 600 tokens — so iteration 3 starts with 1,400 tokens of prior context. By iteration 10, you can be passing 8,000–15,000 tokens of accumulated history into every single call, even if the actual work in each step is small.

Tool outputs compound this further. When an agent calls a function and receives a result, the function schema, the call, and the result are all appended to the message array. A single tool schema can run 200–500 tokens. An agent that calls ten tools accumulates 2,000–5,000 tokens of schema overhead before any reasoning happens — per iteration, not once at session start.

Sub-agent fan-out multiplies the cost again. If an orchestrator spawns five parallel worker agents, each receiving a copy of the shared context, you pay for that shared context five times simultaneously. A 10,000-token shared context sent to five workers in five rounds is 250,000 input tokens from that context alone — before any worker produces useful output.

The cost model varies by provider and changes with each generation of models. For current rates check anthropic.com/pricing and openai.com/api/pricing. What does not change: the ratio between a fast, cheap tier and a full-reasoning tier is typically 10× to 50× on input tokens. A 50-step chain on a premium model can cost 20× more than the same chain properly routed across model tiers — before you factor in context accumulation.

Anatomy of a runaway loop

Runaway loops are not reckless code. They are the rational output of agents that have not been given a way to stop. Three patterns account for the majority of cases.

Pattern 1 — Ambiguous success condition. An agent is tasked with "fix the failing tests." If the agent does not have an explicit exit condition — a deterministic check that the tests pass — it will keep iterating. Without a clear termination signal, "keep trying" is the rational response. The loop is not malicious; it is thorough in a way that costs money.

Pattern 2 — Retry-on-error without escape. An agent that fails on step 3 retries step 3. If the error is structural — wrong tool call schema, insufficient permissions, network timeout that will not resolve — the retry will fail the same way. Without a max-attempt gate and an error-classification step that distinguishes transient from structural failures, the agent retries indefinitely. Each retry passes the full accumulated context as a new input.

Pattern 3 — Unbounded context accumulation. Each tool call result is appended in full to the message array. A debug agent that reads a 400-line log file on each iteration — appending the full log as a tool result each time — adds 600–1,200 tokens per round. Across 20 iterations, the log payload alone contributes 12,000–24,000 tokens of context that grows with every call.

The context growth pattern looks roughly quadratic:

Beyond roughly 50,000 input tokens, research into attention in long contexts — including the "lost in the middle" effects documented across several model families — suggests that retrieval accuracy for specific facts can degrade. You pay more and get worse results from the same model tier when context is bloated.

The fix for each pattern is straightforward once you name it. For Pattern 1: inject an explicit exit check as a tool call — run_tests() → parse result → if PASS, break. For Pattern 2: classify errors before retrying: transient errors (rate limit, 503) get exponential backoff with a max; structural errors (schema invalid, auth denied) halt immediately and escalate. For Pattern 3: summarise tool outputs before appending — keep the diff, the error code, or the key field, not the full 400-line payload.

Hard caps and budget gates

Caps are defensive boundaries you set before the loop starts. Gates are checks you run inside the loop to stop it early. You need both.

Layer 1 — Per-request output cap. Every API call should set max_tokens explicitly. The API default is the model's maximum context — which can be tens or hundreds of thousands of tokens. A reasoning agent does not need 64,000 output tokens per step. For most code-generation or analysis tasks, 2,000–4,000 output tokens per call is sufficient. An explicit cap prevents a single call from producing a document-length response that bloats every subsequent input call.

Layer 2 — Per-session token counter with circuit breaker. Sum usage.input_tokens and usage.output_tokens from each API response. When the session total crosses a threshold, stop the loop and emit a structured summary instead of continuing.

class TokenCircuitBreaker:
    def __init__(self, session_limit: int = 50_000):
        self.session_limit = session_limit
        self.total_tokens = 0

    def record(self, usage) -> None:
        self.total_tokens += usage.input_tokens + usage.output_tokens

    def check(self) -> None:
        if self.total_tokens >= self.session_limit:
            raise TokenBudgetExceeded(
                f"Session budget ({self.session_limit}) reached "
                f"at {self.total_tokens} tokens. "
                "Stopping; emitting partial result."
            )

# Usage in loop:
breaker = TokenCircuitBreaker(session_limit=50_000)
for iteration in range(MAX_ITERATIONS):
    response = client.messages.create(...)
    breaker.record(response.usage)
    breaker.check()          # raises before next iteration
    result = process(response)
    if is_done(result):
        break

The circuit breaker raises — it does not silently swallow. The calling code catches the exception and emits whatever partial result exists. Do not discard the work done so far; discard only the continuation of the loop.

Layer 3 — Per-task budget gate at the orchestration level. Before dispatching a task to an agent, estimate its token budget from the task's scope: file count, diff size, test count. A per-task ceiling set at the orchestrator means that the agent never starts on a task likely to exceed the per-session limit. Start conservative — 10,000 tokens for a single-function refactor; 80,000 for a 10-file migration — instrument your actual runs, then calibrate to P95 of observed usage plus a 20% buffer.

None of these limits are magic numbers pulled from documentation. They come from instrumenting your own agent types against your own codebase. A single-file analysis agent on a 150-line file will have a very different profile than a repository-scan agent reading 200 files. Build the instrumentation first, then set the caps.

Model routing: cheap for easy, premium for reasoning

The most effective cost lever in agentic systems is not the circuit breaker — it is model routing. Most agent calls are not equally hard, and most teams default to the most capable model for every call because it is the safest choice when you have not measured the distribution.

A practical taxonomy for agentic code engineering work:

• Tier 1 — Classification and extraction (~60–70% of calls in a typical agentic workflow): Does this file contain pattern X? Extract all function signatures from this module. Classify this log line as error / warning / info. Parse this JSON schema into a typed structure. These tasks have deterministic-ish outputs and require little reasoning. A fast, cheap model handles them correctly.
• Tier 2 — Synthesis and generation (~25–30% of calls): Write a refactor for this 40-line function given these constraints. Draft a migration for this schema change. Summarise the key changes in this diff. These require generating coherent code or prose and making judgment calls about ambiguity. A mid-tier model is appropriate.
• Tier 3 — Reasoning and design (~5–10% of calls): Analyse the dependency graph of this large codebase. Propose an incremental migration path for this framework transition. Identify the root cause across five interacting failure modes. These tasks genuinely benefit from the strongest available model.

The cost differential across model tiers is significant — the input cost ratio between the cheapest fast tier and a full-reasoning tier typically ranges from 10× to 50× depending on the provider and the model generation. If your agent defaults to the premium tier for every call, and 65% of calls are Tier 1 tasks, you are paying 10–50× the necessary cost on the majority of your workload.

Implementation: classify the task type before dispatching it to the model. You can use a cheap classification call or, in most orchestration architectures, the task type is already known from the work item metadata — use that directly without spending tokens to infer it.

MODEL_TIERS = {
    "tier1": "fast-model",     # classification, extraction, parsing
    "tier2": "balanced-model", # generation, refactoring, summarisation
    "tier3": "reasoning-model" # design, root-cause, architecture
}

TASK_TIER = {
    "classify": "tier1", "extract": "tier1", "parse": "tier1",
    "lint":     "tier1", "diff-summary": "tier1",
    "generate": "tier2", "refactor": "tier2", "summarise": "tier2",
    "design":   "tier3", "architecture": "tier3", "root-cause": "tier3",
}

def select_model(task_type: str) -> str:
    tier = TASK_TIER.get(task_type, "tier2")  # default to balanced
    return MODEL_TIERS[tier]

Replace the placeholder strings with the actual model identifiers for your provider. For Anthropic, current model IDs are listed at docs.anthropic.com/en/docs/about-claude/models; for OpenAI at platform.openai.com/docs/models. These pages change with each new model release — pin the specific version string in your config, not a floating alias like "latest".

A failure mode worth knowing: routing a Tier 3 task to a Tier 1 model to save cost, watching it fail, and then retrying with the full-reasoning model. You pay for both calls and the context accumulated between them. The tier-selection heuristic above is conservative for a reason — when unsure, route up one tier, not down.

Cost attribution and team-level governance patterns make routing decisions auditable across an engineering org — track every call back to a team, a feature, and a routing decision so misroutes show up in the cost report rather than only at the billing-period boundary.

Telemetry that catches it before the bill

The billing cycle is a trailing indicator. By the time an invoice closes, a runaway run is weeks old and the context needed to debug it is gone. Telemetry converts "discover at invoice time" into "catch at runtime."

Every LLM API response exposes usage data. Log it on every call with enough context to attribute cost to the work that generated it:

import logging
import json

def log_api_call(
    *,
    task_id: str,
    agent_type: str,
    model: str,
    usage,
    iteration: int,
) -> None:
    logging.info(json.dumps({
        "event":           "llm_api_call",
        "task_id":         task_id,
        "agent_type":      agent_type,
        "model":           model,
        "iteration":       iteration,
        "input_tokens":    usage.input_tokens,
        "output_tokens":   usage.output_tokens,
        "cached_tokens":   getattr(usage, "cache_read_input_tokens", 0),
    }))

The task_id and agent_type fields are what make the log actionable. Without them, you know your service spent $40 on tokens last Tuesday. With them, you know that task T-4217 (type: repository-scan) consumed $37.60 of that, because the agent iterated 22 times before the session limit fired — and that is a different fix than a general "spend less" directive.

Three alert thresholds that catch most problems before the billing cycle closes:

• Per-task spike. When a single task's accumulated token spend exceeds 2× the rolling P50 baseline for that agent type, flag it for human review before the next iteration runs. This catches runaway loops at iteration 3 or 4, not iteration 40.
• Per-session ceiling warning. When a session crosses 80% of its configured circuit breaker threshold, emit a notification. This gives an engineer time to inspect the session before the breaker fires — useful when the task is close to completion and you want to decide whether to let it continue.
• Daily anomaly alert. Compute a rolling 7-day average of daily token spend per service. When today's spend exceeds the average by more than 2.5 standard deviations, trigger an on-call alert. This catches the pattern where a new agent type was deployed with default-to-premium model routing and nobody reviewed the cost profile before it went live.

If you are using prompt caching — available on several current Anthropic model tiers — the cache_read_input_tokens field shows tokens served from cache at a substantially reduced rate (typically around 10% of the full input cost). Logging this alongside regular input tokens shows you caching effectiveness at a glance. An agent that re-sends the same large system prompt or tool schema on every call without hitting the cache is leaving straightforward savings unused. Prompt caching rates and eligibility are documented at the provider's pricing page.

Putting the four mechanisms together — bounded context per iteration, a per-session circuit breaker, task-tier model routing, and structured telemetry with per-task attribution — gives you the visibility and control needed to run agentic workloads without discovering the cost only when the invoice arrives. Scaled across a team, the same four mechanisms produce per-feature cost attribution — so spend is auditable at the deliverable level, not just at the billing-period boundary.