Five Minutes Or Sixty

Your provider offers two cache lifetimes on the same 30,000-token prefix. A 5-minute write costs 1.25x the base input price, a 1-hour write costs 2x, and a read costs 0.1x either way.

Your traffic is a long tail of agents that come back at irregular intervals. You instrument both settings for a week:

TTL measured cache hit rate
5 minutes 55%
1 hour 97%

Which setting is cheaper per request on the cached prefix, and by roughly how much?

A

The 1-hour TTL, by about 1.3x, since a hit costs 0.1x under both settings and only the misses differ.

B

The 1-hour TTL, by about 4x: 0.157 against 0.618 of the base input price per prefix token.

C

The 5-minute TTL, by about 1.6x, because its write premium is 1.25x rather than 2x.

D

They are within 10 percent of each other, so pick the 5-minute TTL for the lower blast radius on a bad prefix.