Prompt Caching for AI Agents: How to Cut Input Costs

An agent resends almost everything on every turn: the system prompt, every tool definition and the whole conversation so far. On a long task, the same few thousand tokens of instructions get billed dozens of times. Input tokens, not output, often end up as the biggest line on the bill.

Prompt caching fixes most of that. Both Anthropic and OpenAI let you reuse the processed form of a prompt prefix you sent recently, at a fraction of the normal input price. The catch is that the cache only hits if your prompt is built the right way, and agents are very good at accidentally breaking it.

This post covers how caching works, how to structure an agent so it hits, and how to check that it is working.

How prompt caching works

The idea is the same on both platforms. When a request starts with exactly the same tokens as a recent request, the provider can skip reprocessing that shared beginning and charge you less for it.

Two rules follow from that:

Only the prefix counts. The cache matches from the start of the prompt and stops at the first difference. A changed character near the top makes everything after it a miss.
Matches are exact. Anthropic's documentation says cache hits require 100 percent identical prompt segments. "Almost the same" is a miss.

So the whole game is ordering: put what never changes first, and what changes every turn last.

The order that works

For an agent, that usually means:

Tool definitions. Same on every turn.
System prompt. Same on every turn, with no timestamps, user names or request IDs in it.
Stable reference material. Documentation, examples, a codebase summary.
Conversation history. Grows, but earlier turns do not change.
The newest message. Different every time.

On Claude, the API processes the prompt in the order tools, then system, then messages, and a change at one level invalidates the cache for that level and everything after it. Changing a tool definition invalidates everything. Changing the newest message invalidates nothing before it.

Prompt caching on Claude

On the Claude API you mark where cacheable content ends with a cache_control field. Everything up to and including that block becomes a cache entry. Per the prompt caching docs:

You can set up to 4 cache breakpoints per request.
There is also an automatic mode: a single top-level cache_control puts the breakpoint on the last cacheable block and moves it forward as the conversation grows.
The default cache lifetime is 5 minutes, refreshed for free each time the entry is used. A 1-hour lifetime is available with "ttl": "1h".
Writing to the cache costs 1.25 times the base input price for the 5-minute lifetime and 2 times for the 1-hour lifetime. Reading from it costs 0.1 times the base price.
There is a minimum cacheable length, which varies by model (from 1,024 to 4,096 tokens at the time of writing). Shorter prompts are simply not cached, with no error.

Here is an agent turn with two breakpoints: one at the end of the system prompt and one on the latest message, so each turn reuses everything before it.

import anthropic

client = anthropic.Anthropic()
MODEL = "your-claude-model-id"

TOOLS = [...]          # keep this list identical on every turn
SYSTEM_PROMPT = "..."  # no timestamps or per-user text in here

def agent_turn(history: list[dict]) -> anthropic.types.Message:
    # Mark the newest message so the next turn can reuse this whole prefix.
    last = history[-1]
    if isinstance(last["content"], str):
        last["content"] = [{"type": "text", "text": last["content"]}]
    last["content"][-1]["cache_control"] = {"type": "ephemeral"}

    return client.messages.create(
        model=MODEL,
        max_tokens=2048,
        tools=TOOLS,
        system=[
            {
                "type": "text",
                "text": SYSTEM_PROMPT,
                "cache_control": {"type": "ephemeral"},
            }
        ],
        messages=history,
    )

In a real loop, remove the marker from the previous message before adding a new one, so you stay under the 4-breakpoint limit. The docs also note that when a request comes in, the system looks back up to 20 blocks from each breakpoint for an earlier cache entry. In very long conversations, that is a reason to keep a breakpoint near the end rather than only at the top.

Does it pay off?

Treat a normal, uncached prompt as costing 1.0 per use. With the 5-minute cache:

Two uncached uses cost 2.0.
One cache write plus one cache read costs 1.25 + 0.1 = 1.35.

You come out ahead the first time the prefix is reused. With the 1-hour cache, the write costs 2.0, so you need two reuses to come out ahead (2.0 + 0.1 + 0.1 = 2.2, against 3.0 uncached). For an agent that takes ten turns on one task, the shared prefix costs 1.25 + 9 × 0.1 = 2.15 instead of 10, a saving of almost 80 percent.

Prompt caching on OpenAI

OpenAI's prompt caching guide says caching is on by default for supported models, so there is nothing to mark. A few things to know:

Prompts need at least 1,024 tokens to be cached.
The whole prefix has to match, so the same ordering rules apply.
You can pass a prompt_cache_key to group related requests, such as all requests for one user or one task.
Retention options, write pricing and discounts vary by model generation, and newer models add more explicit controls. Check the guide for the model you use rather than relying on a number from a blog post, including this one.

The guide's own advice lines up with everything above: put stable instructions and reference material first, move dynamic content to the end, append to the conversation instead of rewriting earlier turns, and keep tool definitions stable.

Seven ways agents break their own cache

These are the common culprits when the hit rate is lower than it should be:

A timestamp in the system prompt. "Today is November 6, 2026, 3:42 PM" changes every minute. Put the date in the latest user message instead, or round it to the day.
Per-user text near the top. Move names, preferences and account details below the shared instructions.
Changing the tool list between turns. Adding or removing a tool invalidates everything on Claude. Keep the list fixed and restrict tools with tool_choice instead.
Non-deterministic serialization. If tool schemas or context are built from a dictionary with unstable key order, the bytes change even when the content does not. Serialize with sorted keys.
Rewriting history. Summarizing or trimming earlier turns changes the prefix. When you must compact, do it rarely and in large steps, so you pay for one cache write instead of many.
Toggling settings. On Claude, changing tool_choice, extended thinking settings, or adding or removing images invalidates the cached messages.
Gaps longer than the lifetime. If a user pauses for ten minutes, a 5-minute entry is gone. For agents that wait on humans, the 1-hour lifetime can be worth its higher write price.

Measure your hit rate

Do not assume caching works. Read it from the response.

On Claude, the usage object reports three numbers: cache_creation_input_tokens (written to the cache), cache_read_input_tokens (served from it) and input_tokens (processed normally, after the last breakpoint). Their sum is your total input. If both cache fields are zero, nothing was cached, often because the prompt was under the minimum length.

On OpenAI, cached tokens appear in the usage details as cached_tokens.

A simple hit rate for Claude:

def cache_hit_rate(usage) -> float:
    read = usage.cache_read_input_tokens or 0
    written = usage.cache_creation_input_tokens or 0
    fresh = usage.input_tokens or 0
    total = read + written + fresh
    return read / total if total else 0.0

Log it per turn. On a healthy multi-turn agent, the first turn shows mostly writes and later turns should be mostly reads. If later turns keep showing large writes, something near the top of your prompt is changing. Diff two consecutive requests and look for the first byte that differs.

Checklist

Tools first, then system prompt, then history, then the new message.
No timestamps, IDs or user details in the system prompt.
Tool list and schemas byte-for-byte identical across turns.
On Claude: a breakpoint on the system prompt and one on the latest message.
Hit rate logged on every call.

If you are cutting costs, it is also worth checking how much context your tools return on each call. Large tool responses sit in the history and get billed on every later turn. The post on why agents call the wrong tool covers how to keep them short.

Sources

Anthropic, Prompt caching (Claude API documentation)
OpenAI, Prompt caching (API documentation)