We measured 68 cached calls. 22 of them wrote when they should have read.
A third of the cache traffic on one measured workload paid the 1.25x write premium for content that already had a live cache entry — here is what makes a write avoidable, and how to find yours.
When we split one workload's cache traffic into writes and reads for the write-cost measurement, one column demanded a second look. Of 68 calls that touched the cache at all, 22 were writes that should have been reads: the content they wrote was byte-identical to content that already had a live cache entry at the moment they wrote it.
That is not "the cache was cold" or "the prefix changed". It is the narrow, checkable case where everything was in place for a read — same bytes, entry alive, same scope — and the call paid the 1.25x write premium anyway. Roughly a third of the cache traffic on that workload was avoidable spend.
What "avoidable" means, precisely
Calling a write avoidable is a strong claim, so here is the exact test we applied to each of the 22. A write counts as avoidable only when all three hold:
- A prior call had written the same prefix. Same
prefix_hash— the fingerprint the proxy computes over the serialized system prompt and tools — so the bytes were identical, not merely similar. - That entry was still alive. The write landed inside the TTL window of the earlier one. A write 20 minutes after a 5-minute entry expired is a cold-cache write, not an avoidable one, and none of the 22 were that.
- Same cache scope. Same account, same platform, same model. A cache entry warmed on one scope is invisible to another, and a write from a different scope is unavoidable by definition.
Anything failing any test was excluded. What survives is the uncomfortable category: the provider had the bytes, the entry was warm, and the request wrote anyway.
How a warm entry gets re-written
If the bytes match and the entry is alive, why would the API write instead of read? The causes we traced all reduce to the same shape: the request did not present the prefix the way the cache stored it.
The breakpoint moved. A cache entry is keyed by the span up to the cache_control marker, not by the request as a whole. A caller that marks a different block on each call — the whole system prompt on one, only its first paragraph on the next — presents a different cacheable span each time, and a span the cache has not seen is a write, even though the underlying text never changed. We see this most in hand-rolled caching code that computes "where to put the marker" from something that varies, like message count.
The prefix was byte-identical but the span was not. Tools serialize ahead of the system prompt inside the cached span. Two calls with the same system prompt and the same tools in a different order have the same content and different bytes — the case the tool-ordering post covers. On this workload, order flapping between two stable arrangements meant the two variants kept re-writing each other's entries: A writes, B writes, A writes again, every entry alive and none ever read.
Two writers racing. Two instances of the same agent starting within the same second both miss, both write. The second write is avoidable in accounting terms, unavoidable in practice — you would need coordination to stop it, and one duplicated write per cold start is cheap. Only a handful of the 22 were this.
The first two causes are worth engineering time. The third is worth knowing about so you do not chase it.
What it cost
Each avoidable write on this workload was a call in the $0.14–$0.19 range paying the write premium on a span it could have read for roughly a tenth of the price. Against the measured per-call figures, the 22 writes represent about $3 of spend where about $0.30 was available — small in absolute terms because this was one modest agent over one measurement window, and around ten dollars a day of pure waste if you scale it to the agent's monthly volume. On a fleet, the fraction is what matters: a third of the cache traffic was paying full freight next to a warm entry.
The detection matters more than the dollar figure. An agent with a high avoidable-write count looks healthy on every standard metric — it caches, it has entries, its hit rate is non-zero — and it is quietly running at a fraction of its possible saving. Nothing in a provider bill or a default dashboard distinguishes an avoidable write from a necessary one. You have to keep per-prefix history and check each write against it, which is exactly what the proxy's prefix fingerprinting exists to make possible: every call carries a prefix_hash, so "did a live entry exist for these bytes" is a query, not a guess.
What to do
Log the cache-write and cache-read token counts per call if you are not already, and alongside them a hash of the serialized system prompt plus tools — a cheap FNV-1a over the concatenation is enough. Then look for the signature of avoidable writing: the same hash writing repeatedly inside one TTL window. If you find it, check the two usual suspects in order — a cache breakpoint whose position is computed from something that varies per call, and a tool array whose serialization order is not stable. Both fixes are small, and both convert every subsequent write on that prefix into a read at a tenth of the price.