Anthropic and OpenAI batch queues: submit a set of requests, get results within 24 hours (Anthropic: most complete in under an hour), pay half price on every input and output token. Same models, same quality - just async. Batch is not eligible for Zero Data Retention.
The discounts stack: 0.1 (cache-read) × 0.5 (batch) = 0.05 - a 95% discount on a cache-stable prefix processed asynchronously.
from anthropic import Anthropic
client = Anthropic()
batch = client.messages.batches.create(
requests=[
{
"custom_id": f"ticket-{i}",
"params": {
"model": "claude-sonnet-5",
"max_tokens": 200,
"messages": [{"role": "user", "content": ticket}],
},
}
for i, ticket in enumerate(tickets)
]
)
# Poll client.messages.batches.retrieve(batch.id) until ended,
# then iterate client.messages.batches.results(batch.id).When it fits
Evaluation runs, nightly classification, content backfills, bulk labeling - anything that is not waiting on a real-time response. Still trim output; batch does not make prose free.
Worked economics and sibling levers: Stop paying full price for repeat LLM calls.