Blog

Notes from inside the request path

Caching mechanics, cost arithmetic, provider changes that move a bill, and what breaks in a proxy that cannot be allowed to break. Every post is something you can use whether or not you ever point an SDK at us.

RSS · one post a week, Tuesday mornings

Mechanics15 September 20267 min read
A timestamp in your system prompt costs you the whole cache, every call
Prompt caches match on exact bytes, so a clock in your prefix misses every time — and the detector we wrote to catch it was so eager it switched caching off for the word "Tuesday".

A timestamp in your system prompt costs you the whole cache, every call