# Stop paying full price for repeat LLM calls

> Prompt caching cuts 90–97.5% off repeated tokens, batching cuts 50%, and the two stacked cut 95% or more. A 2026 PwC study measured 41–80% real savings from caching alone.

URL: https://bartlomiejkrupa.dev/articles/llm-cost-optimization/
Published: 2026-07-09
Updated: 2026-10-14
Tags: cost, caching, batching, model-routing, context-management, pricing, tokens
Summary: Prompt caching cuts 90–97.5% off repeated tokens, batching cuts 50%, and the two stacked cut 95% or more. A 2026 PwC study measured 41–80% real savings from caching alone.

Prompt caching and batch processing are the two LLM cost levers with numbers you can bank on: Anthropic and OpenAI both publish fixed discount percentages, and the two stack. A cache read costs 90% less than a fresh input token on most models, and 95–97.5% less on the newest Claude tiers. Batch processing cuts every token 50%, in exchange for accepting a result within 24 hours instead of immediately. Apply both to the same request and the discount compounds to 95% at the standard read rate. On Claude Sonnet 5.5, whose cache reads Anthropic halved on 2026-10-07, it reaches 97.5%: $0.05 per million input tokens instead of $2.

Those are list prices. Field data now backs the caching number; the batching and stacked 95% figures are still list-price arithmetic. A January 2026 study by a PwC team ([arXiv 2601.06007](https://arxiv.org/abs/2601.06007)) measured prompt caching across OpenAI, Anthropic, and Google on real agent workloads: 41–80% off total API cost. The same research also found the catch this article keeps returning to. The cache only reuses tokens that are byte-identical to a prior call - its "prefix" - and one careless byte breaks that match. A separate 2026 study ([arXiv 2607.12161](https://arxiv.org/abs/2607.12161)) caught the failure mode in production: trimming tokens, the "obvious" lever, raised the bill 6.8% by breaking exactly that prefix.

## Definitions

**Prompt caching** - storing the processed representation of a repeated prompt prefix so later calls skip recomputing it, paying a steep discount on cache reads instead of the full input price.

**Prompt prefix** - the front portion of a request (tool schemas, system prompt, early conversation history) that stays byte-identical across calls. Caching matches on the prefix, so any change to it invalidates everything after the change.

**Batch processing** - submitting requests for asynchronous handling, with results delivered within a fixed turnaround window (an SLA, short for service-level agreement) instead of immediately, in exchange for a flat discount on every token.

**Model-tier routing** - sending each request to the cheapest model tier that can handle it, reserving the most expensive frontier tier for genuinely hard tasks.

**Context management** - deliberately curating what stays in an agent's conversation history each turn instead of letting it grow unbounded; every token in history is resent, and repaid, on every subsequent call.

**Cache breakpoint** - the marked position where a cacheable prefix ends. Everything before it can be written to and read from the cache; everything after it bills as fresh input.

## The levers at a glance

| Lever | Headline number | Source |
| --- | --- | --- |
| Prompt caching | 90–97.5% off cache reads (list); 41–80% measured on agent workloads | Anthropic docs; [arXiv 2601.06007](https://arxiv.org/abs/2601.06007) |
| Batch processing | 50% off all tokens, 24h SLA | Anthropic + OpenAI batch docs |
| Caching + batching stacked | 95–98.75% off cacheable async work (list; batch cache hits are best-effort) | Arithmetic: 0.1 × 0.5 (0.05 × 0.5 on Opus 5.5 and Sonnet 5.5, 0.025 × 0.5 on Fable 5.1) |
| Model-tier routing | 100:1 input-price spread across Claude tiers; 13–21% of spend recovered by a cache-safe router | [Claude tier guide](/articles/claude-model-tier-comparison); [arXiv 2609.28919](https://arxiv.org/abs/2609.28919) |
| Context management | 50%+ agent cost cut | JetBrains Research, SWE-bench Verified (via [Prune the log, not the window](/articles/llm-context-management)) |

## Current per-token pricing

Anthropic, OpenAI, and DeepSeek all publish a cached-input price alongside base input. The usual discount is 90%. The newest Claude top and mid tiers, GPT-6.1 Sol, and DeepSeek discount further:

| Model | Input | Cached input read | Output |
| --- | --- | --- | --- |
| Claude Fable 5.1 | $10.00 | $0.25 | $50.00 |
| Claude Fable 5 | $10.00 | $1.00 | $50.00 |
| Claude Opus 5.5 | $4.00 | $0.20 | $20.00 |
| Claude Opus 5 | $5.00 | $0.50 | $25.00 |
| Claude Sonnet 5.5 | $2.00 | $0.10 | $10.00 |
| Claude Sonnet 5 | $2.00 | $0.20 | $10.00 |
| Claude Haiku 5.5 (prompt up to / over 100K) | $0.10 / $0.50 | $0.01 / $0.05 | $0.50 / $2.50 |
| Claude Haiku 4.5 | $1.00 | $0.10 | $5.00 |
| GPT-6 Astra | $10.00 | $1.00 | $50.00 |
| GPT-6.1 Sol | $2.00 | $0.10 | $10.00 |
| GPT-6 Luna | $0.10 | $0.01 | $0.50 |
| GPT-5.5 | $5.00 | $0.50 | $30.00 |
| DeepSeek V4 Pro (off-peak / peak) | $0.66 / $1.32 | $0.022 / $0.044 | $1.98 / $3.96 |
| DeepSeek V4.1 Flash (off-peak / peak) | $0.15 / $0.30 | $0.003 / $0.006 | $0.60 / $1.20 |

Prices per million tokens, re-verified 2026-10-14. What changed and what to watch:

- **2026-10-07, Anthropic:** Claude Haiku 5.5 launched at a tenth of Haiku 4.5's price, and Sonnet 5.5's cache reads dropped from $0.20 to $0.10.
- **Haiku 5.5 bills by prompt length.** A request over 100,000 input tokens pays the higher rates on every token in it, cache reads included.
- **OpenAI:** the GPT-6 lineup shipped on 2026-09-22 and GPT-6.1 Sol a week later. The prices above cover prompts up to 272K tokens; longer prompts pay long-context rates.
- **DeepSeek bills by time of day.** Peak hours (01:00–04:00 and 06:00–10:00 UTC on weekdays) cost double the off-peak rate.
- **The 90% cache-read discount is still the market pattern.** It holds for Fable 5, Opus 5, Sonnet 5, both Haikus, GPT-6 Astra and Luna, and GPT-5.5. The exceptions: Opus 5.5, Sonnet 5.5, and GPT-6.1 Sol read cache at 0.05x base input (95% off), and Fable 5.1 at 0.025x (97.5% off).
- **Sonnet 5's $2/$10 is its standard price.** The move to $3/$15 announced for 2026-09-01 did not happen.

## Lever 1: Prompt caching - 90% or more off repeated tokens

Anthropic's prompt caching stores a marked prefix (system prompt, reference docs, tool schemas) and charges a fraction of the base input price on every call that reuses it: 0.1x on most models (a 90% discount), 0.05x on Opus 5.5 and Sonnet 5.5, 0.025x on Fable 5.1. The first call pays a write premium instead: 1.25x base for a 5-minute cache, 2x base for a 1-hour cache. After that, every reused token costs the read rate.

```python
from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,  # marked once, reused on every call
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": user_query}],
)
```

OpenAI caches without code changes, but where the cached prefix ends now depends on the model. Before GPT-5.6, caching is implicit and writes are free; the minimum cacheable length varies with request settings. On GPT-5.6 and later, including the whole GPT-6 lineup, a write costs 1.25x base input (matching Anthropic), the minimum is 1,024 visible tokens, and an entry lives at least 30 minutes after its last use. The default implicit breakpoint sits at the end of the latest message. If that message changes on every call, OpenAI writes a cache entry nobody reuses, and you pay the write premium for it. Put an explicit breakpoint after the stable prefix, and switch to explicit-only mode when the changing tail isn't worth writing:

```json
{
  "model": "gpt-6.1-sol",
  "prompt_cache_options": { "mode": "explicit" },
  "input": [
    {
      "role": "developer",
      "content": [
        {
          "type": "input_text",
          "text": "Stable instructions and shared reference material...",
          "prompt_cache_breakpoint": { "mode": "explicit" }
        }
      ]
    },
    { "role": "user", "content": "The user's current question..." }
  ]
}
```

### Measured, not promised

The 90% figure (and 95–97.5% on the newest tiers) is a list price; what you actually save depends on how much of each request is a reused prefix. The PwC study put a number on it. Across 500+ agent sessions with 10,000-token system prompts, on a multi-turn research benchmark, caching cut API cost 41–80% and made the first token of each response arrive 13–31% sooner (time-to-first-token), on all three providers tested (OpenAI, Anthropic, Google). The spread is the prefix share: sessions that reuse more context land near 80%, sessions dominated by fresh tokens near 41%.

### Don't break the cache

The cache is a prefix match on exact bytes. A timestamp interpolated into the system prompt, a reordered tool list, a request ID near the top - any of these invalidates every cached token after it, silently. You still get a response; you just pay full price plus the write premium, every call.

Three placement rules, straight from the same study's findings - the garden rule [Cache-stable prefix](/notes/cache-stable-prefix) is the one-liner:

- **Stable content first, dynamic content last.** Frozen system prompt and tool schemas before the cache marker; timestamps, user IDs, and per-request context after it.
- **Exclude dynamic tool results from the cached block.** Caching them writes a new cache entry every turn instead of reading the old one.
- **Don't cache everything by reflex.** The study found naive full-context caching can paradoxically *increase* latency versus caching only the stable system prompt.

Two more silent failure modes worth checking. First, Anthropic's minimum cacheable prefix is model-dependent, and older models are the trap: 512 tokens on every current Claude model (Fable 5 and 5.1, Opus 5 and 5.5, Sonnet 5.5, Haiku 5.5), 1,024 on Sonnet 5 and Opus 4.8, 2,048 on Opus 4.7, and 4,096 on Haiku 4.5, Opus 4.5, and Opus 4.6. A prefix under the model's floor won't cache at all, with no error. A 3,000-token policy prefix caches fine on Haiku 5.5 and silently never caches on Haiku 4.5, so a pipeline still pinned to the old cheap tier pays full input price on every call. Check the floor for the model you routed to, not the one you developed against.

Second, the cache expires after its TTL (time-to-live) window if nothing reuses it - 5 minutes or 1 hour on Anthropic, at least 30 minutes on OpenAI's GPT-5.6 and later. Caching a prefix you call once is a write premium with no read to offset it.

The first check is one field: if `usage.cache_read_input_tokens` is zero across repeated calls, something in your prefix is changing per request. Anthropic's cache diagnostics then tells you what: pass the previous response's `id` as `diagnostics.previous_message_id`, and the response names the first place the two prompts diverged - model, system prompt, tools, or message history.

## Lever 2: Batch processing - 50% off, if you can wait

Both Anthropic and OpenAI run a separate batch queue: submit a set of requests, get results back within 24 hours (Anthropic's docs note most complete in under an hour), and pay half price on every input and output token. No quality difference - same models, same weights, just asynchronous delivery. See [batch submit pattern](/notes/batch-submit-pattern).

```python
from anthropic import Anthropic

client = Anthropic()

batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": f"ticket-{i}",
            "params": {
                "model": "claude-sonnet-5-5",
                "max_tokens": 200,
                "messages": [{"role": "user", "content": ticket}],
            },
        }
        for i, ticket in enumerate(tickets)
    ]
)
```

Batching only works where nothing is waiting on the response in real time: evaluation runs, nightly classification jobs, content backfills, bulk data labeling. Anthropic's batch queue is not eligible for Zero Data Retention (Anthropic's no-logging guarantee for sensitive data), so check that constraint before routing anything sensitive through it.

OpenAI also sells batch rates without the batch queue. Set `service_tier` to `flex` on a normal Responses or Chat Completions call and tokens bill at Batch API prices, in exchange for slower responses and occasional `429 Resource Unavailable` errors, which aren't charged. Flex is in beta on a limited set of models; it fits evals and enrichment jobs that want a synchronous call shape.

## Stacked: caching and batching compound to 95% or more

The two discounts are independent multipliers, so a cached prefix processed through the batch queue pays both at once: 0.1 (cache) × 0.5 (batch) = 0.05, a 95% discount off the base input price. With the 0.05x reads on Opus 5.5 and Sonnet 5.5 it is 0.05 × 0.5 = 0.025 (97.5% off); on Fable 5.1, 0.025 × 0.5 = 0.0125 (98.75% off).

Worked example, Claude Sonnet 5.5 ($2.00/M input, $0.10/M cache read, $2.50/M 5-minute cache write, $10.00/M output), a nightly job classifying 10,000 support tickets against a shared 4,000-token policy prefix, ~200 unique input tokens and 50 output tokens per ticket. Batch halves every rate in the bottom two rows:

| | Prefix | Unique input | Output | Total / ticket |
| --- | --- | --- | --- | --- |
| No optimization | 4,000 × $2.00/M = $0.0080 | 200 × $2.00/M = $0.0004 | 50 × $10.00/M = $0.0005 | $0.0089 |
| Cached + batched, every prefix a cache hit | 4,000 × $0.05/M = $0.0002 | 200 × $1.00/M = $0.0002 | 50 × $5.00/M = $0.00025 | $0.00065 |
| Cached + batched, 30% cache hits | 4,000 × (0.3 × $0.05 + 0.7 × $1.25)/M = $0.00356 | 200 × $1.00/M = $0.0002 | 50 × $5.00/M = $0.00025 | $0.00401 |

With every prefix read from cache, per-ticket cost drops from $0.0089 to $0.00065 - $89.00/day to $6.50/day at 10,000 tickets, a 92.7% cut on the full request. The shared prefix gets the full 97.5% discount; the unique per-ticket tokens only get the 50% batch rate, since they're never cached.

That row is the ceiling, not the forecast. Batch requests run asynchronously and concurrently, so Anthropic serves cache hits inside a batch on a best-effort basis and reports typical hit rates of 30% to 98%. Every miss pays the batched cache-write rate ($1.25/M on Sonnet 5.5) instead of the read rate. At 30% hits the same job costs $40.10/day, a 55% cut. To push toward the top of that range, Anthropic's batch docs recommend identical `cache_control` blocks in every request of the batch and a steady stream of requests, so entries don't expire inside their 5-minute lifetime. This is illustrative arithmetic built on the sourced prices above, not a benchmark - your split between shared and unique tokens, and your hit rate, will change the exact percentage.

## Lever 3: Model-tier routing

Cost per input token now varies 100:1 across the current Claude stack: Haiku 5.5 at $0.10/M versus Fable 5.1 at $10/M. Routing routine requests to a cheaper tier and reserving the frontier tier for genuinely hard tasks is a model-selection problem, covered in full in [Pick the right Claude tier for agent workflows](/articles/claude-model-tier-comparison).

The cheap end moved on 2026-10-07. Haiku 5.5 costs a tenth of Haiku 4.5 for prompts up to 100,000 tokens; Anthropic puts the average saving at about 75%, after counting its new tokenizer, which spends more tokens on the same text. Above 100,000 tokens the whole request bills at 5x the base rate, so a sub-agent whose context grows past that line loses most of the discount. GPT-6 Luna matches Haiku 5.5 at $0.10/$0.50 and steps up only past 272K tokens, to $0.20/$0.75.

Routing also has a cache cost the per-token price hides. A prompt cache belongs to one model, so switching models mid-conversation rebuilds the cache at the write premium. A September 2026 paper on enterprise coding-agent harnesses ([arXiv 2609.28919](https://arxiv.org/abs/2609.28919)) built a router that switches models only where no running conversation loses its cache: at session start, on separate side tasks, and when a sub-agent launches. In a simulation of a 10,000-employee company, it recovered an estimated 13–21% of model spend. The same paper found that on long, tool-heavy sessions the most expensive model can be the cheapest. On Anthropic's 21 September price sheet, Fable 5.1's $0.25/M cache reads beat Opus 5's $0.50/M once context passed about 211k tokens at six tool steps per turn, because cache reads dominate those bills. That specific crossover predates Opus 5.5, which undercuts Fable 5.1 on every rate. The rule outlasts it: for agent workloads, compare models on their cache-read rate, not their input price.

## Lever 4: Context management

Every token in an agent's conversation history gets resent, and repaid, on every subsequent call - a long-running session pays for its early turns dozens of times over. [Prune the log, not the window](/articles/llm-context-management) covers JetBrains Research's SWE-bench Verified findings in full: observation masking and LLM summarization both cut agent cost over 50% versus letting context grow unmanaged, with masking also improving solve rate rather than just holding it steady.

## Lever 5: Trim output, guard the prefix

Cost is `(input tokens × input price) + (output tokens × output price)` - there is no minimum charge, so fewer tokens always means a smaller bill. But the two sides of that formula are not equal, and one of them can bite back.

**Output is the expensive side.** On every Claude tier above, output tokens cost 5x input tokens (6x on GPT-5.5), and output is never cached: every generated token bills at full rate, forever. That makes output the best place to cut - structured output instead of free-form prose, explicit length limits, no restated question in the answer. It also scales: CROP ([arXiv 2604.14214](https://arxiv.org/abs/2604.14214)) regularized response length during prompt optimization and cut token consumption 80.6% with only a nominal accuracy dip on GSM8K, LogiQA, and BIG-Bench Hard.

**Input trimming can backfire.** "Token Reduction Is Not Cost Reduction" ([arXiv 2607.12161](https://arxiv.org/abs/2607.12161)) measured 2,848 provider-billed Claude Code runs and found that the test setup removing 38% of raw tool-output tokens paid 6.8% *more*, not less. The mechanism: prompt-cache traffic made up ~87% of total cost, and compressing content mid-session rewrote bytes inside the cached prefix - turning cheap cache reads back into full-price input. The compression also corrupted the verbatim text the agent's edit tool matched against, dropping successful patch application from 27/40 to 15/40. Worse output, higher bill.

The rule that reconciles both findings: **trim what's never cached, and never rewrite inside a cached prefix mid-session.** A bloated-but-stable system prompt costs 0.1x or less after the first call; verbose free-form output costs 5x input on every call. As illustrative arithmetic on Sonnet 5.5 pricing: 300 calls/day through a cached 1,500-token system prompt cost about $0.045/day in cache reads, while the same 300 calls each padding their answer with 300 tokens of prose cost $0.90/day in output. The padding costs twenty times the "bloat."

## Where to start

Work the levers in order of effort-to-certainty:

1. **Cache the stable prefix** - a `cache_control` field on Anthropic (top-level for automatic caching, or on the last stable block), automatic on OpenAI with an explicit `prompt_cache_breakpoint` after the stable prefix on GPT-5.6 and later - and structure requests so the prefix stays byte-stable: fixed content first, dynamic content last, dynamic tool results excluded.
2. **Batch everything that can wait**, or use OpenAI's `flex` tier for the same rate on a synchronous call. A queue instead of a real-time call is worth a flat 50%, and it stacks with caching as long as the batch keeps the cache warm.
3. **Route by tier** for real-time traffic that caching can't help - the [Claude tier guide](/articles/claude-model-tier-comparison) covers the 100:1 spread. Switch models at session or sub-agent boundaries, never mid-conversation.
4. **Manage agent context** so history stops compounding - [Prune the log, not the window](/articles/llm-context-management) has the sourced numbers.
5. **Trim output last**, with the prefix rule above in mind.

And measure while you do it: break token spend down by model and by feature, and tie it to a business number - cost per resolved ticket, per processed document - so each lever's effect shows up somewhere other than the invoice. The first two levers are configuration, not architecture; if your workload has a repeated prefix or a tolerance for delay, that's the same code you already have, at a fraction of the price. Sibling playbooks in this cluster: [Don't break the prompt cache](/articles/dont-break-the-prompt-cache), [Trim output, not the cache](/articles/trim-output-not-the-cache), and [Five metrics for an agent eval pipeline](/articles/five-metrics-for-an-agent-eval-pipeline). Before you optimize a closed API bill, decide whether that path should stay closed at all - [Open-weights ≠ open-source for agent stacks](/articles/open-weights-vs-closed-api). Hub: [Optimization](/topics/optimization).

Sources:
- Prompt caching - cache write and read multipliers, minimum cacheable length, TTL: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- Batch processing - 50% discount, 24-hour turnaround, best-effort cache hits: https://platform.claude.com/docs/en/build-with-claude/batch-processing
- Pricing - per-model input, cached-read, and output rates: https://platform.claude.com/docs/en/about-claude/pricing
- Introducing Claude Haiku 5.5 - Haiku 5.5 pricing and the Sonnet 5.5 cache-read cut: https://www.anthropic.com/claude-haiku-5-5
- Cache diagnostics - find where a prompt prefix diverged: https://platform.claude.com/docs/en/build-with-claude/cache-diagnostics
- Prompt caching - implicit and explicit breakpoints, write and read multipliers, minimum length: https://developers.openai.com/api/docs/guides/prompt-caching
- Flex processing - Batch API rates on synchronous requests: https://developers.openai.com/api/docs/guides/flex-processing
- Pricing - per-model input, cached input, cache writes, and output: https://developers.openai.com/api/docs/pricing
- Models & Pricing - peak and off-peak rates: https://api-docs.deepseek.com/quick_start/pricing
- Prompt caching on agent workloads: 41–80% cost reduction (arXiv 2601.06007): https://arxiv.org/abs/2601.06007
- Token Reduction Is Not Cost Reduction (arXiv 2607.12161): https://arxiv.org/abs/2607.12161
- CROP: response-length regularization during prompt optimization (arXiv 2604.14214): https://arxiv.org/abs/2604.14214
- Cutting Through the Noise: Smarter Context Management for LLM-Powered Agents: https://blog.jetbrains.com/research/2025/12/efficient-context-management/
- Harness Tokenomics: a router for the enterprise agentic control plane (arXiv 2609.28919): https://arxiv.org/abs/2609.28919