# Trim output, not the cache

> Output tokens cost ~5× input and never cache - cut free-form prose first. Rewriting inside a cached prefix mid-session can raise the bill, not lower it.

URL: https://bartlomiejkrupa.dev/articles/trim-output-not-the-cache/
Published: 2026-07-20
Updated: 2026-09-30
Tags: cost, tokens, output, caching, agents
Summary: Output tokens cost ~5× input and never cache - cut free-form prose first. Rewriting inside a cached prefix mid-session can raise the bill, not lower it.

Cost is `(input × input price) + (output × output price)`. There is no minimum charge, so fewer tokens usually means a smaller bill - except when the cut lands inside a cached prefix. Then you can pay more for less text.

This is the trim-side playbook that pairs with [Don't break the prompt cache](/articles/dont-break-the-prompt-cache). Pricing tables and the full five-lever map live in [Stop paying full price for repeat LLM calls](/articles/llm-cost-optimization).

## Definitions

**Cached prefix** - the byte-stable front of a request that later calls reuse at ~0.1× input price.

**Output budget** - an explicit max length or schema for the model's reply so it cannot pad with restated questions, apologies, or essay structure.

**Mid-session rewrite** - changing text that already sat inside a cached block (for example compressing older tool output in place) so later turns miss the cache and pay full input again.

## The asymmetry

| Side | Relative price (Claude) | Cached? | Trim priority |
| --- | --- | --- | --- |
| Stable system / tool prefix | 0.1× after first write | Yes | Keep stable; do not rewrite |
| Unique per-call input | 1× | No | Trim only if it never enters a cache block |
| Model output | ~5× input | Never | Cut first |

Illustrative arithmetic on Sonnet 5 pricing ($2/$10) from the cost article: 300 calls/day through a cached 1,500-token system prompt cost about $0.09/day in cache reads, while the same 300 calls each padding the answer with 300 tokens of prose cost $0.90/day in output. The padding costs ten times the "bloat."

The garden rule [Output tokens never cache](/notes/output-tokens-never-cache) is the one-liner version of this asymmetry.

## What the studies measured

**Output-side trimming works.** In the CROP prompt-optimization study ([arXiv 2604.14214](https://arxiv.org/abs/2604.14214)), regularizing response length cut token consumption 80.6% with only a nominal accuracy dip on GSM8K, LogiQA, and BIG-Bench Hard.

**Input trimming can backfire.** "Token Reduction Is Not Cost Reduction" ([arXiv 2607.12161](https://arxiv.org/abs/2607.12161)) measured 2,848 provider-billed Claude Code runs. An arm that removed 38% of raw tool-output tokens paid 6.8% *more* (95% confidence interval: +2.8% to +11.3%). Prompt-cache traffic was ~87% of input-side cost, and the compressed runs took extra retrieval, diagnosis, and testing turns that ate the savings. Compression also corrupted the verbatim text the edit tool matched against, dropping successful patches from 27/40 to 15/40.

## Copy-paste: output budget

```python
# Prefer a schema + hard max_tokens over "be concise" prose.
response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=80,  # hard ceiling - cheapest control you have
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,  # stable; see cache playbook
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[
        {
            "role": "user",
            "content": (
                "Classify the ticket.\n"
                "Reply exactly: LABEL | one-line reason\n"
                "Labels: billing, outage, feature, other\n\n"
                f"{ticket_text}"
            ),
        }
    ],
)
```

Rules that survive contact with production:

1. Cap `max_tokens` to the longest useful answer, not the model default.
2. Demand a schema (`LABEL | reason`, JSON, enum) so the model cannot invent essay structure.
3. Never compress or rewrite text that already sits inside a cached prefix mid-session - mask or drop older turns instead ([Prune the log, not the window](/articles/llm-context-management)).
4. If you must slim tool output, do it **before** it enters a cacheable block, or leave it uncached entirely ([Cache-stable prefix](/notes/cache-stable-prefix)).

## Exit

Apply the output budget on one noisy endpoint this week, keep the prefix byte-stable, and compare `output_tokens` and `cache_read_input_tokens` before/after. For evals that catch silent quality regressions while you cut spend, use [Five metrics for an agent eval pipeline](/articles/five-metrics-for-an-agent-eval-pipeline). Hub: [Optimization](/topics/optimization).

Sources:
- Token Reduction Is Not Cost Reduction: https://arxiv.org/abs/2607.12161
- CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization: https://arxiv.org/abs/2604.14214
- Pricing: https://platform.claude.com/docs/en/about-claude/pricing
- API pricing: https://developers.openai.com/api/docs/pricing