Trim output, not the cache

Cost is (input × input price) + (output × output price). There is no minimum charge, so fewer tokens usually means a smaller bill - except when the cut lands inside a cached prefix. Then you can pay more for less text.

This is the trim-side playbook that pairs with Don’t break the prompt cache. Pricing tables and the full five-lever map live in Stop paying full price for repeat LLM calls.

Definitions

Cached prefix - the byte-stable front of a request that later calls reuse at ~0.1× input price.

Output budget - an explicit max length or schema for the model’s reply so it cannot pad with restated questions, apologies, or essay structure.

Mid-session rewrite - changing text that already sat inside a cached block (for example compressing older tool output in place) so later turns miss the cache and pay full input again.

The asymmetry

Illustrative arithmetic on Sonnet 5 intro pricing from the cost article: 300 calls/day through a cached 1,500-token system prompt cost about $0.09/day in cache reads, while the same 300 calls each padding the answer with 300 tokens of prose cost $0.90/day in output. The padding costs ten times the “bloat.”

What the studies measured

Output-side trimming works. In the CROP prompt-optimization study (arXiv 2604.14214), regularizing response length cut token consumption 80.6% with only a nominal accuracy dip on GSM8K, LogiQA, and BIG-Bench Hard.

Input trimming can backfire. “Token Reduction Is Not Cost Reduction” (arXiv 2607.12161) measured 2,848 provider-billed Claude Code runs. An arm that removed 38% of raw tool-output tokens paid 6.8% more (95% confidence interval: +2.8% to +11.3%). Prompt-cache traffic was ~87% of cost. Compression also corrupted the verbatim text the edit tool matched against, dropping successful patches from 27/40 to 15/40.

Copy-paste: output budget

# Prefer a schema + hard max_tokens over "be concise" prose.
response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=80,  # hard ceiling - cheapest control you have
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,  # stable; see cache playbook
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[
        {
            "role": "user",
            "content": (
                "Classify the ticket.\n"
                "Reply exactly: LABEL | one-line reason\n"
                "Labels: billing, outage, feature, other\n\n"
                f"{ticket_text}"
            ),
        }
    ],
)

Rules that survive contact with production:

1. Cap max_tokens to the longest useful answer, not the model default.
2. Demand a schema (LABEL | reason, JSON, enum) so the model cannot invent essay structure.
3. Never compress or rewrite text that already sits inside a cached prefix mid-session - mask or drop older turns instead (Prune the log, not the window).
4. If you must slim tool output, do it before it enters a cacheable block, or leave it uncached entirely (Cache-stable prefix).

Exit

Apply the output budget on one noisy endpoint this week, keep the prefix byte-stable, and compare output_tokens and cache_read_input_tokens before/after. For evals that catch silent quality regressions while you cut spend, use Five metrics for an agent eval pipeline. Hub: Optimization.