Don't break the prompt cache

Prompt caching is a 90% list-price discount on reused input tokens - but only while the prefix stays byte-identical across calls. A single interpolated timestamp, a reordered tool list, or a request ID near the top invalidates every cached token after it. You still get a response; you just pay full price plus the write premium, every call.

This playbook is the operational half of Stop paying full price for repeat LLM calls. Numbers and provider pricing live there; the checklist and verification snippet live here.

Definitions

Prompt prefix - the front portion of a request (system prompt, tool schemas, early history) that the provider matches byte-for-byte on a cache hit.

Cache marker - the boundary you set (Anthropic cache_control) or the provider sets (OpenAI auto-cache ≥1,024 tokens) between content eligible for reuse and content that changes every call.

Cache read - a later call that reuses a stored prefix at ~0.1× base input price (90% off on Anthropic and the GPT-5.x tiers covered in the cost article).

Placement rules

Three rules, grounded in the same PwC study that measured 41–80% field savings from caching (arXiv 2601.06007):

Anthropic’s minimum cacheable prefix is model-dependent, and the ordering is the opposite of the intuition - newer and pricier models have the lower floor:

Shorter prefixes silently don’t cache - no error, just zero reads. The practical trap is routing: a 3,000-token prefix caches on Opus 5 and does not on Haiku 4.5, so the moment you fan work out to the cheap tier to save money, that tier starts paying full input price on every call. Check the floor for the model you route to, not the one you developed against. Every model’s floor, plus the invalidation hierarchy and the 20-block lookback window, is in Prompt cache floors by model. Cache time-to-live (TTL) also matters: a prefix you call once pays the write premium with no read to offset it.

You get four breakpoints, and they are a budget. Anthropic allows a maximum of 4 cache_control breakpoints per request, counted across tools + system + messages combined - and automatic caching consumes one of the slots. Exceed the limit and the request fails outright with a 400, which is the good outcome; the bad outcome is a long-running agent loop that keeps appending breakpoints as history grows and hits the ceiling mid-session.

Caching also matches on everything up to and including a marked block, in tools → system → messages order. So spend the budget from the front: one breakpoint after tool schemas, one after the system prompt, and give a rolling message window whatever remains. An agent loop that marks each new turn is not caching more aggressively - it is running out of budget and re-writing the prefix.

Copy-paste: Anthropic stable prefix

from anthropic import Anthropic

client = Anthropic()

# STABLE - byte-identical across calls in this session
SYSTEM_PROMPT = """You are a ticket classifier.
Allowed labels: billing, outage, feature, other.
Reply with one label and a one-line reason.
"""

TOOLS = [
    {
        "name": "lookup_policy",
        "description": "Fetch a frozen policy snippet by id",
        "input_schema": {
            "type": "object",
            "properties": {"policy_id": {"type": "string"}},
            "required": ["policy_id"],
        },
    }
]  # freeze order; do not rebuild per request

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=64,
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,
            "cache_control": {"type": "ephemeral"},
        }
    ],
    tools=TOOLS,  # keep schemas above any per-call noise
    messages=[
        {
            "role": "user",
            # DYNAMIC - after the cached block
            "content": f"[{request_id}] {ticket_text}",
        }
    ],
)

usage = response.usage
print(
    usage.cache_creation_input_tokens,
    usage.cache_read_input_tokens,
    usage.input_tokens,
)

OpenAI caches automatically on prompts ≥1,024 tokens - same placement discipline applies even without a cache_control block.

Verify before you celebrate

Fire the same request twice and assert the second call reads the cache. Anything else is guessing. The garden note Cache-read verifier is this check in one place.

def check_cache(build_request, n=2):
    """Call the same request n times; the prefix is healthy if call 2+ reads."""
    for i in range(1, n + 1):
        u = client.messages.create(**build_request()).usage
        print(
            f"call {i}: write={u.cache_creation_input_tokens} "
            f"read={u.cache_read_input_tokens} uncached={u.input_tokens}"
        )
        if i > 1:
            assert u.cache_read_input_tokens > 0, "prefix changed or is under the minimum"
            assert u.cache_creation_input_tokens == 0, "prefix is being rewritten every call"


check_cache(lambda: dict(
    model="claude-sonnet-5",
    max_tokens=64,
    system=[{"type": "text", "text": SYSTEM_PROMPT,
             "cache_control": {"type": "ephemeral"}}],
    tools=TOOLS,
    messages=[{"role": "user", "content": "ping"}],
))

Healthy output writes on call 1 and reads on call 2:

call 1: write=1532 read=0    uncached=9
call 2: write=0    read=1532 uncached=9

When reads stay at zero, diff the two payloads instead of rereading the code - the byte that moved is rarely the one you suspect:

import json, difflib

a = json.dumps(build_request(), indent=2, sort_keys=True).splitlines()
b = json.dumps(build_request(), indent=2, sort_keys=True).splitlines()
print("\n".join(difflib.unified_diff(a, b, lineterm="")))

Building the request twice and diffing it catches the whole class in one shot: interpolated timestamps, a regenerated request ID, a tool list rebuilt from a dict in nondeterministic order, a session UUID in the system prompt. An empty diff with zero reads means the opposite problem - the prefix is stable but under the model’s minimum, so raise it or accept that this endpoint will not cache.

The garden rule Cache-stable prefix is the one-liner version of this section.

Stack, don’t substitute

Caching stacks with batch processing (95% combined on the shared prefix) and with context management on agent loops. It does not survive mid-session rewrites of cached content - that failure mode is covered in Trim output, not the cache. For tier routing instead of cache math, use the Claude tier guide.

Next move: apply the checklist on one high-volume endpoint, log cache_read_input_tokens for a day, then expand. Full lever map: Optimization.