Prompt caching and the economics of context

Here is the cost structure nobody warns you about: a production Claude system re-sends the same system prompt, the same tool definitions, and the same conversation history on every request. A 50-turn agent session can easily bill millions of input tokens, and almost all of them are bytes the API has already seen dozens of times. Prompt caching is the fix — cached input is served at roughly a tenth of the normal input price — and it is the single largest cost lever available to you. But it only works if you understand one rule.

Everything follows from the prefix rule

Caching is an exact prefix match. Any changed byte invalidates everything after it. The request renders in a fixed order — tools, then system, then messages — and the cache key is the literal bytes of that rendering up to a marker you place. Change a character at position N and every cached byte after N is worthless.

This is why the earlier lessons kept insisting on discipline you couldn't yet see the reason for: the frozen system prompt, the stable tool list. The architecture rule is simply: stable content first, volatile content last. Fixed instructions and tools at the front; per-session context next; the user's newest message, timestamps, and per-request state at the very end, after the last cache marker.

Placing the marker

You opt in by placing cache_control on the last block of the stable prefix:

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    system=[{
        "type": "text",
        "text": BIG_STABLE_SYSTEM_PROMPT,        # instructions + docs + examples
        "cache_control": {"type": "ephemeral"},   # cache everything up to here
    }],
    messages=[{"role": "user", "content": user_question}],
)

Because tools render before system, this one marker caches tools and system together. For multi-turn conversations, also mark the last content block of the newest turn — each request then reuses the entire prior conversation as cached prefix, and the cache grows with the session. Up to four markers are allowed per request; the simplest strategy — top-level cache_control on the request, which auto-marks the last cacheable block — is enough for most applications.

The economics, so you can do the arithmetic

A cache write costs a premium — about 1.25× normal input price for the default 5-minute lifetime, 2× for the 1-hour lifetime — and every read costs about 0.1×. So a prefix read twice within the window has already paid for itself, and an agent loop that re-reads its prefix thirty times pays roughly a tenth of what it would uncached. Two fine-print items: each cache lifetime refreshes on use, so steady traffic keeps entries warm for free; and very short prefixes are silently not cached at all — minimums are model-specific, on the order of a few hundred to a few thousand tokens.

The silent invalidators

Caching fails without errors — you just quietly pay full price. When usage.cache_read_input_tokens is zero across requests that should share a prefix, one of these is almost always the culprit:

• A timestamp or UUID in the system prompt. f"Today is {date.today()}" at the top invalidates everything under it, every request. Move dynamic context into a late message.
• Non-deterministic serialization. json.dumps without sort_keys=True, iterating a set to build tool lists — same data, different bytes, no match.
• A tool set that varies per user or per request. Tools render at position zero; any variation destroys the whole cache. Keep one canonical, sorted tool list.
• Switching models mid-conversation. Caches are per-model. Route to different models across sessions, not within one.

The verification habit: read usage on every response. cache_read_input_tokens high and input_tokens low is the shape of a healthy system; anything else, diff two rendered requests byte-by-byte and the invalidator will be staring at you.

What to take into the next lesson

One rule — prefix match — one placement pattern — stable first, marker at the boundary — and one metric — cache_read_input_tokens. Cost drops of 60–90% on context-heavy workloads are normal, which is why cache discipline was baked into the prompt lessons before you knew to ask. Next: the delivery side — streaming, and the stop reasons that tell you how a response actually ended.