Cutting agent tokens can raise your bill
A 2,848-run study of Claude Code found prompt-cache traffic, not raw token count, drives most billed cost, and one compression technique made costs go up.
A new study analyzing 2,848 real, provider-billed Claude Code runs across 103 tasks, seven repositories, and three models found that cutting the number of tokens an agent sends does not reliably cut what you actually pay. The researchers ran a pre-specified, hash-frozen campaign comparing an uncompressed baseline against two generations of context-compression techniques and an API-boundary proxy.
The headline finding: prompt-cache operations, not raw token volume, account for roughly 80-87% of real billed cost, with about 8.7% left unexplained. Against that backdrop, one compression technique that cut estimated tool-output tokens by 38% actually raised paired cost by 6.8%, because it disrupted cache hit patterns that would otherwise have kept the bill down.
The technique also broke things. On Go-language tasks, patch-application success dropped from 27 of 40 to 15 of 40, because the compression removed context the agent needed as edit anchors, meaning fewer completed tasks at a higher cost per completed task. The authors argue that any context-reduction layer should be evaluated on “success-adjusted billed cost,” not token count in isolation, since token-count reduction can look like a win on a dashboard while making both the bill and the outcome worse.
For anyone running Claude Code or a similar coding agent at scale, this is a concrete warning against adopting a context-compression setting purely because it advertises fewer tokens per call. Test it against your own cache-hit patterns and task success rate first; a smaller prompt is not automatically a cheaper or more effective one.