Prompt Caching ROI: When It Pays Off, When It Does

Anthropic introduced prompt caching in August 2024 and OpenAI matched in October 2024. The mechanics are simple, the pricing is more complex, and the break-even math is rarely written down. This article covers the math, the hit-rate sensitivity, and the workload shapes where caching wins or loses.

1. What prompt caching actually is

Prompt caching is a server-side cache of input-token activations keyed by a prefix hash. When you send a request with a marked cacheable prefix, the provider checks whether that exact prefix has been seen recently. If yes (a hit), the cached activations are reused and you are charged a discounted rate for those tokens. If no (a miss), the activations are computed and cached, and you are charged a premium rate for those tokens to cover the write.

The key parameters at Anthropic^[1]:

• Minimum prefix size: 512 tokens on Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5; older models need 1,024 to 4,096. Below the minimum the request is processed without caching and no error is returned.
• TTL: 5 minutes default, extendable to 1 hour on Anthropic with a higher write multiplier. Cache entries are evicted on TTL or under memory pressure.
• Granularity: Caching is prefix-only. Anything before the cache breakpoint is cached; anything after is not. Order matters: stable content first, variable content last.
• Pricing: A 5-minute cache write costs 1.25x the base input rate and a 1-hour write 2x. A cache read costs 0.1x, 0.05x, 0.025x of the base rate depending on the model.

OpenAI's current models price cached input at 5% to 10% of the base rate and cache writes at 1.25x^[3]. Google Gemini supports explicit context caching with a per-token storage charge billed by the hour^[6]. The break-even math below uses Anthropic's rates; the same logic applies at OpenAI and Google with their write and read rates.

2. Pricing mechanics: write, read, miss

Anthropic's current rates per million tokens, from the model catalog the hub's calculators share^[2]:

Model              Base input   5-min write   1-hour write   Cache read   Output
Claude Fable 5.1   $10.00/M     $12.500/M     $20.00/M       $0.250/M     $50.00/M
Claude Opus 5.5    $4.00/M      $5.000/M      $8.00/M        $0.200/M     $20.00/M
Claude Sonnet 5.5  $2.00/M      $2.500/M      $4.00/M        $0.100/M     $10.00/M
Claude Haiku 5.5   $0.10/M      $0.125/M      $0.20/M        $0.010/M     $0.50/M

Three operations occur on a request with caching enabled:

• Cache miss with write: The marked prefix is not in cache. You pay the cache-write rate (1.25x base for the 5-minute cache) for those tokens, and the prefix is stored.
• Cache hit: The marked prefix is in cache. You pay the cache-read rate (0.05x base on Claude Sonnet 5.5, 0.1x on Claude Haiku 5.5) for those tokens.
• Cache miss without write: Same as a regular API call with no caching. You pay the base input rate.

The write premium and the read discount are the two numbers that drive the break-even calculation. The read rate is what makes caching attractive; the write rate is what creates the break-even threshold.

3. Break-even derivation

Let P be the cost of a regular (uncached) input. Let W be the write cost (1.25xP on Claude Sonnet 5.5). Let R be the read cost (0.05xP on Claude Sonnet 5.5). For N total requests with cache enabled, where the first request is always a write and the remaining N-1 may be hits or misses, hit-rate H gives expected cost:

Cost(cached) = W + (N-1) * (H * R + (1-H) * W)

The uncached baseline is simply Cost(uncached) = N * P. Caching pays off when Cost(cached) < Cost(uncached), which simplifies to:

1.25x + (N-1) * (0.05x·H + 1.25x·(1-H)) < N

Solving for the break-even hit count at H = 1.0 (perfect hits, no eviction):

1.25x + (N-1) * 0.05x < N → N > 1.26

At H = 1.0 (no eviction during the TTL window), caching wins from the second request. The first request pays a 25% premium; every request after it pays 5% of the base rate.

At H = 0.5 the break-even is 2 requests; at H = 0.3 it is 4 requests. Below a hit rate of about 21% caching never pays: each later request then costs more on average than an uncached one, because the write premium is paid more often than the read discount earns it back.

4. Hit-rate sensitivity table

The break-even request count and the saving for different hit rates, on Claude Sonnet 5.5 pricing (1.25x write, 0.05x read) and a stable workload pattern:

Hit rate H    Break-even N    Net savings at N=10    Net savings at N=100
1.0 (ideal)   2 requests      83% reduction          94% reduction
0.9           2 requests      72% reduction          82% reduction
0.7           2 requests      51% reduction          58% reduction
0.5           2 requests      29% reduction          34% reduction
0.3           4 requests      7% reduction           11% reduction
0.1           never           14% surcharge          13% surcharge

The takeaway. Above a hit rate of about 21% caching eventually pays, and from 30 percent it pays within a handful of requests; below it, caching is a net loss. A stable system prompt reused inside the TTL sits near the top of this table; per-user state above the cache breakpoint pushes a workload to the bottom.

Hit rate is reported in the Anthropic API response under usage.cache_read_input_tokens and usage.cache_creation_input_tokens^[5]. Aggregate these per route weekly to know your actual hit rate; do not rely on intuition about cache effectiveness.

5. Which workload shapes win

Caching wins decisively for these workload shapes:

• RAG with stable corpus. A retrieval-augmented assistant where 80% of the prompt is a fixed system prompt + retrieved documents that change slowly. Cache the system prompt + the top-K retrieved chunks per session; most requests in a session hit.
• Few-shot examples. A classification or extraction prompt with 5-20 stable few-shot examples followed by the variable input. Cache the few-shot block; nearly every request hits while the examples stay the same.
• Long context coding. A coding assistant where the user's repo (or a relevant subset) is in context. Cache the repo content; requests hit within a session and miss once the user returns after the TTL.
• Document Q&A with a single source. Cache the document; ask multiple questions against it within the TTL window. Hit rate approaches 100% for the question-asking phase.

Caching loses or breaks even for these shapes:

• One-shot generation. A user submits a single prompt and never returns within the TTL. The write premium is paid; no read discount is earned. Net loss of 25%.
• Highly personalized prefixes. The system prompt includes per-user state above the cache breakpoint. Cache writes happen per user, hit rate is 0% across users, only sequential requests by the same user hit. Marginal value at best.
• Prefixes shorter than the minimum. Below 512 tokens on the current Claude models (1,024 to 4,096 on older ones), caching is not eligible. The request is processed uncached and no error is returned.
• Workloads with TTL-aligned bursts. A nightly batch job where a 50,000-token system prompt is reused across 1,000 requests in a 30-minute window. Hit rate is high but the workload pattern fits the 1-hour TTL extension better; the 5-minute default forces re-writes.

6. Caching anti-patterns

The four most common caching mistakes in production:

• Caching the wrong layer. Marking the entire prompt as cacheable, including the user input. Anthropic only caches the prefix up to the breakpoint; everything before the breakpoint must be byte-identical to hit. If the user input is in the prefix, hit rate drops to 0%.
• Caching too small a prefix. Marking a 400-token prefix on a current Claude model (which requires 512 tokens). The marker is silently ignored, no caching occurs, and the developer thinks caching is enabled.
• Caching variable content. Putting today's date, a session ID, or a request ID near the start of the prompt. Even one variable byte invalidates the prefix hash. Move all variable content after the cache breakpoint.
• Caching across model versions. Cache entries are keyed per model. Switching from claude-3-5-sonnet-20240620 to claude-sonnet-4-20250514 mid-rollout invalidates all cached entries and requires re-writes.

Measure the hit rate per route (section 8). A team that enables caching once and never measures cannot tell whether its breakpoint is placed well, and the sensitivity table shows how much a low hit rate gives back.

7. Worked example: a docs assistant

A solo founder builds a documentation assistant. The product loads a 25,000-token system prompt (instructions + 10 few-shot examples + retrieved doc snippets) and answers user questions in 200-token outputs. The product handles 50,000 user queries per month, with average session length of 4 questions and average inter-question gap of 90 seconds.

Without caching on Claude Sonnet 5.5:

Cost = 50,000 queries * 25,000 input tokens * $2.00/M = $2,500.00/month input

With caching enabled on the system prompt (the 25,000-token prefix is identical across queries within a session, but the next session in 5+ minutes is a fresh write):

• Each session of 4 queries has 1 write + 3 hits.
• 50,000 queries / 4 queries-per-session = 12,500 sessions/month.
• Writes: 12,500 * 25,000 tokens * $2.50/M = $781.25/month
• Reads: 37,500 * 25,000 tokens * $0.10/M = $93.75/month
• Total input cost with caching: $875.00/month
• Savings vs uncached baseline: 65% ($1,625.00/month)

The hit rate in this example is 75% (3 hits out of 4 requests per session). Sessions whose queries all fall inside the 5-minute TTL get every follow-up as a hit; sessions that span TTL boundaries pay extra writes and drop the hit rate accordingly.

A 1-hour cache write costs 2x the base input rate ($4.00/M on Claude Sonnet 5.5). It pays when the same prefix is reused across sessions or after gaps longer than five minutes within the hour, so the extra write premium buys hits the 5-minute cache would have missed.

8. The implementation checklist

Six steps to implement caching profitably:

1. Measure your current input-token cost per route. If a route consumes more than $20/month and has a stable prefix above the minimum size, caching is worth implementing.
2. Identify the largest stable prefix in each route. System prompt + few-shot examples + retrieved documents are the typical candidates.
3. Place the cache breakpoint after the stable prefix, before any variable content (user input, today's date, session ID).
4. Deploy and measure hit rate for one week using cache_read_input_tokens and cache_creation_input_tokens from the API response.
5. If hit rate is below 50%, debug. Common causes: variable content above the breakpoint, prefix too small, model version mismatch, sessions too short for the TTL.
6. If your sessions span more than 5 minutes, evaluate the 1-hour TTL extension. The extra write premium is worth it only if your steady-state hit rate inside the window is above 70%.

Prompt caching is one of the few API features where the ROI math is unambiguous and the implementation cost is hours, not days. The read discount is large enough that any production workload with a stable prefix above the model's minimum reused twice or more per TTL window saves money. The risk is implementing caching badly: marking a 500-token prefix that does not qualify, putting variable content above the breakpoint, or failing to measure the hit rate after deployment. Measure before, measure after, and tune the breakpoint until hit rate is above 70%.

Frequently asked questions

When does prompt caching pay off?

When the same prefix is reused often enough that the read savings cover the write premium. On Anthropic a 5-minute cache write costs 1.25x the base input rate and a cache read costs 0.1x, 0.05x, 0.025x of it depending on the model (0.05x on Claude Sonnet 5.5), so at a perfect hit rate caching wins from the second request. It pays for stable prefixes above the model's minimum cacheable length (512 tokens on the current Claude models) reused inside the 5-minute TTL, and never for one-shot prompts or prefixes that change on every request.

How much does prompt caching reduce cost?

It depends on hit rate. On Claude Sonnet 5.5 pricing, ten requests sharing a prefix cost 51% reduction at a 70 percent hit rate, 72% reduction at 90 percent and 83% reduction at a perfect hit rate, against sending the prefix uncached every time. Below a hit rate of about 21% caching never pays, because the write premium is paid more often than the read discount earns it back; at 10 percent it is a 14% surcharge.

What are the most common prompt caching mistakes?

Four dominate: caching the wrong layer (marking the user input as cacheable, so any change breaks the prefix and the hit rate drops to zero), caching too small a prefix (below the model's minimum, 512 tokens on the current Claude models and 1,024 to 4,096 on older ones, where the marker is ignored without an error), caching variable content (a date, session ID or request ID near the start invalidates the prefix; move all variable content after the breakpoint), and caching across model versions (cache entries are kept per model, so a model switch starts every cache from a write).

How does OpenAI caching differ from Anthropic's?

OpenAI's current models price cached input at 5% to 10% of the base input rate and cache writes at 1.25x of it. Anthropic prices a 5-minute cache write at 1.25x, a 1-hour write at 2x and a read at 0.1x, 0.05x, 0.025x depending on the model, with explicit cache breakpoints. Google Gemini supports explicit context caching with a per-token storage charge billed hourly. The same break-even logic applies to all three; plug each provider's write and read rates into it.