A timestamp kept the prompt out of the cache.
At 100,000 replies a month, that is about $220 a month.
OpenAI caches an identical prompt prefix of at least 1,024 tokens. In Captain's instructions, the current time (to the minute) and the conversation context sat ahead of about 1,411 tokens of fixed rules, so the stable part ended after ~544 tokens and nothing was cached. The second call of a reply already cached fine.
The fix moves the time and the context block to the end of the instructions, in two templates. The text stays the same; only its position changes.
The prompt is small (~2,000 tokens), so the saving is small in dollars. We also measured the obvious alternative, setting prompt_cache_key: it made results worse, so the report advises against it.
The same thing in your product
Anything that changes with each request, such as a timestamp, an ID or a user name, stops caching for everything that comes after it in the prompt. It is the most common leak we find: we found it in PostHog and Metabase too. If your prompts are long and mostly fixed, it is the first thing we check.