Skip to content

Browse catalog

Search catalog

[READY] Type a title, tag, or description.

NOTES // rule

Output tokens never cache

LAST_MODIFIED
2026.10.07
CATEGORY
rule

Output tokens bill at full price every call and never enter a prompt cache. On current Claude tiers they cost about 5x input (6x on GPT-5.5). That makes them the first place to cut when spend rises - not the stable prefix you depend on for cache hits.

Why trimming input can backfire

A 2026 study of 2,848 provider-billed Claude Code runs (arXiv 2607.12161) found that removing 38% of raw tool-output tokens raised total cost 6.8%: cache traffic was ~87% of input-side cost, so compression could reach only a small slice of the bill, and it changed the agent’s trajectory with extra retrieval and testing turns that each re-sent the cached prefix. The rule: trim what is never cached; never rewrite inside a cache-stable prefix mid-session.

Rule

  1. Cap max_tokens to the longest useful answer.
  2. Demand a schema (LABEL | reason, JSON, enum) so the model cannot invent essay structure.
  3. Mask or drop older turns instead of compressing text already inside a cached block.
  4. If you must slim tool output, do it before it enters a cacheable block.

Illustrative Sonnet 5 arithmetic ($2/$10): 300 calls/day through a cached 1,500-token system prompt cost ~$0.09/day in cache reads; the same 300 calls each padding 300 tokens of prose cost ~$0.90/day in output.

Playbook: Trim output, not the cache. Pair with Don’t break the prompt cache.

Sources

  1. Token Reduction Is Not Cost ReductionarXiv
  2. PricingAnthropic
  3. API pricingOpenAI
costtokensoutputcaching

Related_Notes