Trim output, not the cache

Cost is (input × input price) + (output × output price). There is no minimum charge, so fewer tokens usually means a smaller bill - except when the cut lands inside a cached prefix. Then you can pay more for less text.

This is the trim-side playbook that pairs with Don’t break the prompt cache. Pricing tables and the full five-lever map live in Stop paying full price for repeat LLM calls.

Definitions

Cached prefix - the byte-stable front of a request that later calls reuse at ~0.1× input price.

Output budget - an explicit max length or schema for the model’s reply so it cannot pad with restated questions, apologies, or essay structure.

Mid-session rewrite - changing text that already sat inside a cached block (for example compressing older tool output in place) so later turns miss the cache and pay full input again.

The asymmetry

Illustrative arithmetic on Sonnet 5 pricing ($2/$10) from the cost article: 300 calls/day through a cached 1,500-token system prompt cost about $0.09/day in cache reads, while the same 300 calls each padding the answer with 300 tokens of prose cost $0.90/day in output. The padding costs ten times the “bloat.”

The garden rule Output tokens never cache is the one-liner version of this asymmetry.

What the studies measured

Output-side trimming works. In the CROP prompt-optimization study (arXiv 2604.14214), regularizing response length cut token consumption 80.6% with only a nominal accuracy dip on GSM8K, LogiQA, and BIG-Bench Hard.

Input trimming can backfire. “Token Reduction Is Not Cost Reduction” (arXiv 2607.12161) measured 2,848 provider-billed Claude Code runs. An arm that removed 38% of raw tool-output tokens paid 6.8% more (95% confidence interval: +2.8% to +11.3%). Prompt-cache traffic was ~87% of input-side cost, and the compressed runs took extra retrieval, diagnosis, and testing turns that ate the savings. Compression also corrupted the verbatim text the edit tool matched against, dropping successful patches from 27/40 to 15/40.

Copy-paste: output budget

# Prefer a schema + hard max_tokens over "be concise" prose.
response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=80,  # hard ceiling - cheapest control you have
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,  # stable; see cache playbook
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[
        {
            "role": "user",
            "content": (
                "Classify the ticket.\n"
                "Reply exactly: LABEL | one-line reason\n"
                "Labels: billing, outage, feature, other\n\n"
                f"{ticket_text}"
            ),
        }
    ],
)

Rules that survive contact with production:

1. Cap max_tokens to the longest useful answer, not the model default.
2. Demand a schema (LABEL | reason, JSON, enum) so the model cannot invent essay structure.
3. Never compress or rewrite text that already sits inside a cached prefix mid-session - mask or drop older turns instead (Prune the log, not the window).
4. If you must slim tool output, do it before it enters a cacheable block, or leave it uncached entirely (Cache-stable prefix).

Exit

Apply the output budget on one noisy endpoint this week, keep the prefix byte-stable, and compare output_tokens and cache_read_input_tokens before/after. For evals that catch silent quality regressions while you cut spend, use Five metrics for an agent eval pipeline. Hub: Optimization.