Who Holds The Lock

To stop a stampede you add a distributed lock: on a miss, one caller acquires a per-key lock with a 10 second expiry and queries the origin, and the others sleep and retry the cache. The key is read 20,000 times per second and the origin call takes about 900 milliseconds. What is the real weakness of this design?

Retrying waiters with exponential backoff and jitter fully removes the synchronized wake-up, so the design is sound once that backoff is added

If the holder stalls or dies, waiters pile up for the full lock lifetime and then release as one wave, so the herd is postponed rather than removed

The lock removes every duplicate origin query, so the only cost left is a modest rise in tail latency for the waiters that sleep and retry the cache

Holding a per-key lock across the origin call blocks writes to unrelated keys on the same node, which serializes the whole cache cluster