Shipping: cost, latency, and reliability

A prototype has to work once; production has to work at p99, under load, at a price that survives contact with the finance team. The good news: everything in this lesson is an application of tools you already hold. This is the assembly manual — the levers in the order they pay, the platform decision most teams meet late, and the checklist to run before real users arrive.

Cost: pull the levers in this order

1. Cache the prefix. The caching lesson's discipline — frozen system prompt, stable tools, marker at the boundary — is worth more than every other optimisation combined on context-heavy workloads. Verify with cache_read_input_tokens, not with faith.

2. Right-size the model per route. Your evaluation suite tells you where Sonnet or Haiku holds quality; adjacent tiers differ roughly 2× in price. A router pattern — Haiku classifies the request, then dispatches to the cheapest capable model — is often a lunch-break implementation with a 60% bill reduction.

3. Batch everything that can wait. The Message Batches API runs asynchronous workloads at half price — nightly enrichment, backfills, evaluation runs, bulk classification. Submit up to 100,000 requests, poll, collect results within hours (results return keyed by your IDs, in no particular order — design for that). If a human is not waiting on the response, it belongs in a batch.

4. Control output length. Output tokens cost ~5× input. Concise-output instructions, structured outputs instead of prose, and per-route effort tuning all trim the expensive side of the meter.

Latency: what users actually feel

Users feel time to first token, and you already own the three levers that move it: stream everything user-facing (the first token arrives in moments even when the full answer takes thirty seconds); cached prefixes process faster as well as cheaper — one more return on the same discipline; and smaller models start faster, which is one reason the router pattern helps latency, not just cost. For multi-step flows, look for parallelism you are leaving on the table — independent calls should be concurrent, and Claude's parallel tool calls (returned together, as the tools lesson insisted) already batch what naive loops serialize.

Reliability: assume every failure will happen

The SDK retries transient failures — 429s, 5xxs, dropped connections — with exponential backoff out of the box; your job is the layer above. Respect retry-after on rate limits and know your tier's requests-per-minute and tokens-per-minute ceilings before launch day discovers them for you. Branch on stop_reason everywhere — max_tokens truncation and refusal are normal states, and each needs a designed path, not a stack trace. Log request_id and usage on every call: the first is your lifeline with support, the second your cost telemetry. And decide your degradation story now — queue, retry with backoff, fall back to a smaller model, or fail honestly — because "the API had a bad minute" is an eventuality, not a hypothesis.

Where you run it: the platform decision

Claude is available beyond Anthropic's first-party API: through Amazon Bedrock, Google Vertex AI, and Microsoft Foundry, where billing rides your existing cloud agreement and data stays inside your cloud perimeter — often the deciding factor for procurement and compliance. The trade to price in: the first-party API gets new models and features first and carries the fullest feature surface (some server-side tools and beta features land on partner platforms later or not at all), and model IDs and client setup differ slightly per platform. The pragmatic default: build against the first-party API; move or dual-home when procurement, data residency, or committed cloud spend says so — and if you do, re-run your evaluation suite there, because a platform is part of the system under test. Enterprise controls — SSO, usage tiers, priority capacity for latency-sensitive workloads, and stricter data-retention arrangements — are decisions to raise before launch, not after the first compliance review; note that retention agreements interact with model choice (the most capable models can require standard retention).

The pre-launch checklist

• Evaluation suite green, wired into CI, seeded with your ugliest real cases.
• Cache hit rate verified in usage on realistic traffic — not assumed.
• Every route on the cheapest model your evals clear; batch-eligible work actually batched.
• stop_reason branches: truncation retried or surfaced, refusals handled honestly, pause_turn resumed.
• Streaming on every user-facing path; timeouts and retries configured; rate-limit headroom checked against expected load.
• Platform, data-retention, and capacity decisions made deliberately — not inherited from the prototype.
• request_id, usage, and tool-call traces logged; cost per route on a dashboard someone looks at.

What to take into the next lesson

Cost levers in order, latency levers users feel, reliability as designed paths rather than surprises, and the platform decision made on purpose. One column of the checklist got a placeholder: the security items. They deserve more than a bullet — they get the next lesson.