Cache Hit Rate Is the Cost Lever Your Team Is Probably Ignoring I have watched teams spend a month on model selection benchmarks — GPT-4o versus Claude Sonnet 4.5 versus Gemini 2.5 Pro — then deploy with a prompt structure that breaks cache hits on every single request, paying three to five times more than they should for work the provider has already done. The model selection decision is worth something. The prompt structure decision is worth more. For any workload with repeated or agentic patterns, it is not close. The mechanism is the KV cache. Every major LLM API — Anthropic, OpenAI, Google — reuses computation when a new request begins with tokens it has already processed. Anthropic charges cache reads at 10 percent of the standard input price . A clean cache hit is a 90 percent discount on those tokens. The catch is that cache hits only fire when the prefix — the exact sequence of tokens at the start of your request — matches what was previously cached.…