Stop paying full price for repeat LLM calls

Prompt caching and batch processing are the two LLM cost levers with numbers you can bank on: Anthropic and OpenAI both publish fixed discount percentages, and the two stack. A cache read costs 90% less than a fresh input token. Batch processing cuts every token 50%, in exchange for accepting a result within 24 hours instead of immediately. Apply both to the same request and the discount compounds to 95%: on Claude Sonnet 5 (currently $2/M input on intro pricing), that’s $0.10 per million tokens instead of $2.

Those are list prices; field data now backs the caching number specifically - batching and the stacked 95% figure remain list-price arithmetic. A January 2026 study by a PwC team (arXiv 2601.06007) measured prompt caching across OpenAI, Anthropic, and Google on real agent workloads: 41–80% off total API cost. The same research also found the catch this article keeps returning to. The cache only reuses tokens that are byte-identical to a prior call - its “prefix” - and one careless byte breaks that match. A separate 2026 study (arXiv 2607.12161) caught the failure mode in production: trimming tokens, the “obvious” lever, raised the bill 6.8% by breaking exactly that prefix.

Definitions

Prompt caching - storing the processed representation of a repeated prompt prefix so later calls skip recomputing it, paying a steep discount on cache reads instead of the full input price.

Prompt prefix - the front portion of a request (tool schemas, system prompt, early conversation history) that stays byte-identical across calls. Caching matches on the prefix, so any change to it invalidates everything after the change.

Batch processing - submitting requests for asynchronous handling, with results delivered within a fixed turnaround window (an SLA, short for service-level agreement) instead of immediately, in exchange for a flat discount on every token.

Model-tier routing - sending each request to the cheapest model tier that can handle it, reserving the most expensive frontier tier for genuinely hard tasks.

Context management - deliberately curating what stays in an agent’s conversation history each turn instead of letting it grow unbounded; every token in history is resent, and repaid, on every subsequent call.

The levers at a glance

Current per-token pricing

Anthropic, OpenAI, and DeepSeek all publish a cached-input price alongside base input. As of today that’s a consistent 90% discount for every model below, except DeepSeek, which discounts even further:

Prices per million tokens, current as of 2026-07-17. The 90% cache-read discount is a market pattern, not one vendor’s promotion - it holds across every Claude and GPT-5.x tier above. Sonnet 5 is mid-intro-pricing: the $2/$10 rate is in effect only through 2026-08-31, so anything computed against it needs a re-check after that date.

Lever 1: Prompt caching - 90% off repeated tokens

Anthropic’s prompt caching stores a marked prefix (system prompt, reference docs, tool schemas) and charges 0.1x the base input price on every call that reuses it - a 90% discount. The first call pays a write premium instead: 1.25x base for a 5-minute cache, 2x base for a 1-hour cache. After that, every reused token costs 10% of the base price.

from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,  # marked once, reused on every call
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": user_query}],
)

OpenAI’s caching is automatic on prompts of 1,024 tokens or more - no cache_control block required, and no code changes. Models before GPT-5.6 pay no write fee at all; GPT-5.6 and later pay a 1.25x write premium, matching Anthropic’s structure.

Measured, not promised

The 90% figure is a list price; what you actually save depends on how much of each request is a reused prefix. The PwC study put a number on it. Across 500+ agent sessions with 10,000-token system prompts, on a multi-turn research benchmark, caching cut API cost 41–80% and improved time-to-first-token 13–31%, on all three providers tested (OpenAI, Anthropic, Google). The spread is the prefix share: sessions that reuse more context land near 80%, sessions dominated by fresh tokens near 41%.

Don’t break the cache

The cache is a prefix match on exact bytes. A timestamp interpolated into the system prompt, a reordered tool list, a request ID near the top - any of these invalidates every cached token after it, silently. You still get a response; you just pay full price plus the write premium, every call.

Three placement rules, straight from the same study’s findings:

• Stable content first, dynamic content last. Frozen system prompt and tool schemas before the cache marker; timestamps, user IDs, and per-request context after it.
• Exclude dynamic tool results from the cached block. Caching them writes a new cache entry every turn instead of reading the old one.
• Don’t cache everything by reflex. The study found naive full-context caching can paradoxically increase latency versus caching only the stable system prompt.

Two more silent failure modes worth checking. Anthropic’s minimum cacheable prefix is model-dependent (1,024–4,096 tokens depending on the model): a shorter prefix won’t cache at all, with no error. And the cache expires after its TTL (time-to-live) window if nothing reuses it, so caching a prefix you call once is a write premium with no read to offset it. The verification is one field: if usage.cache_read_input_tokens is zero across repeated calls, something in your prefix is changing per request.

Lever 2: Batch processing - 50% off, if you can wait

Both Anthropic and OpenAI run a separate batch queue: submit a set of requests, get results back within 24 hours (Anthropic’s docs note most complete in under an hour), and pay half price on every input and output token. No quality difference - same models, same weights, just asynchronous delivery.

from anthropic import Anthropic

client = Anthropic()

batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": f"ticket-{i}",
            "params": {
                "model": "claude-sonnet-5",
                "max_tokens": 200,
                "messages": [{"role": "user", "content": ticket}],
            },
        }
        for i, ticket in enumerate(tickets)
    ]
)

Batching only works where nothing is waiting on the response in real time: evaluation runs, nightly classification jobs, content backfills, bulk data labeling. Anthropic’s batch queue is not eligible for Zero Data Retention (Anthropic’s no-logging guarantee for sensitive data), so check that constraint before routing anything sensitive through it.

Stacked: caching and batching compound to 95%

The two discounts are independent multipliers, so a cached prefix processed through the batch queue pays both at once: 0.1 (cache) × 0.5 (batch) = 0.05, a 95% discount off the base input price.

Worked example, Claude Sonnet 5 at current intro pricing ($2.00/M input, $10.00/M output), a nightly job classifying 10,000 support tickets against a shared 4,000-token policy prefix, ~200 unique input tokens and 50 output tokens per ticket:

Per-ticket cost drops from $0.0089 to $0.00085 - $89.00/day to $8.50/day at 10,000 tickets, roughly a 90% cut on the full request. The shared prefix hits the full 95% discount; the unique per-ticket tokens only get the 50% batch rate, since they’re never cached. This is illustrative arithmetic built on the sourced prices above, not a benchmark - your split between shared and unique tokens will change the exact percentage, and the same math on standard Sonnet 5 pricing after 2026-08-31 scales up proportionally (the percentages hold, the dollar figures don’t).

Lever 3: Model-tier routing

Cost per token varies up to 10:1 across the current Claude stack: Haiku 4.5 at $1/M input versus Fable 5 at $10/M. Routing routine requests to a cheaper tier and reserving the frontier tier for genuinely hard tasks is a pure model-selection problem, covered in full in Pick the right Claude tier for agent workflows: Fable 5 as a rare, hardest-step escalation, Opus 4.8 as orchestrator, Sonnet 5 as the default worker, Haiku 4.5 fanned out across high-volume, low-complexity subtasks run in parallel.

Lever 4: Context management

Every token in an agent’s conversation history gets resent, and repaid, on every subsequent call - a long-running session pays for its early turns dozens of times over. Prune the log, not the window covers JetBrains Research’s SWE-bench Verified findings in full: observation masking and LLM summarization both cut agent cost over 50% versus letting context grow unmanaged, with masking also improving solve rate rather than just holding it steady.

Lever 5: Trim output first, guard the prefix

Cost is (input tokens × input price) + (output tokens × output price) - there is no minimum charge, so fewer tokens always means a smaller bill. But the two sides of that formula are not equal, and one of them can bite back.

Output is the expensive side. On every Claude tier above, output tokens cost 5x input tokens (6x on GPT-5.5), and output is never cached: every generated token bills at full rate, forever. That makes output the best place to cut - structured output instead of free-form prose, explicit length limits, no restated question in the answer. It also scales: CROP (arXiv 2604.14214) regularized response length during prompt optimization and cut token consumption 80.6% with only a nominal accuracy dip on GSM8K, LogiQA, and BIG-Bench Hard.

Input trimming can backfire. “Token Reduction Is Not Cost Reduction” (arXiv 2607.12161) measured 2,848 provider-billed Claude Code runs and found that an arm removing 38% of raw tool-output tokens paid 6.8% more, not less. The mechanism: prompt-cache traffic made up ~87% of total cost, and compressing content mid-session rewrote bytes inside the cached prefix - turning cheap cache reads back into full-price input. The compression also corrupted the verbatim text the agent’s edit tool matched against, dropping successful patch application from 27/40 to 15/40. Worse output, higher bill.

The rule that reconciles both findings: trim what’s never cached, and never rewrite inside a cached prefix mid-session. A bloated-but-stable system prompt costs 0.1x after the first call; verbose free-form output costs 5x input on every call. As illustrative arithmetic on Sonnet 5 intro pricing: 300 calls/day through a cached 1,500-token system prompt cost about $0.09/day in cache reads, while the same 300 calls each padding their answer with 300 tokens of prose cost $0.90/day in output. The padding costs ten times the “bloat.”

Where to start

Work the levers in order of effort-to-certainty:

1. Cache the stable prefix - a cache_control block on Anthropic, automatic on OpenAI - and structure requests so the prefix stays byte-stable: fixed content first, dynamic content last, dynamic tool results excluded.
2. Batch everything that can wait. A queue instead of a synchronous call is worth a flat 50%, and it stacks with caching.
3. Route by tier for real-time traffic that caching can’t help - the Claude tier guide covers the 10:1 spread.
4. Manage agent context so history stops compounding - Prune the log, not the window has the sourced numbers.
5. Trim output last, with the prefix rule above in mind.

And measure while you do it: break token spend down by model and by feature, and tie it to a business number - cost per resolved ticket, per processed document - so each lever’s effect shows up somewhere other than the invoice. The first two levers are configuration, not architecture; if your workload has a repeated prefix or a tolerance for delay, that’s the same code you already have, at a fraction of the price. Sibling playbooks in this cluster: Don’t break the prompt cache, Trim output, not the cache, and Five metrics for an agent eval pipeline. Hub: Optimization.