2 / 6 Sharding Strategies & Shard Keys

Fifty Doors One Answer

A search API fans a query out to all 50 shards in parallel and merges the results. Each shard independently shows a median of 8 ms and a p99 of 40 ms, yet the end-to-end p99 measured at the coordinator is well above 100 ms. What explains the gap?

A

Fifty concurrent connections exhaust the coordinator's connection pool, so most requests spend their time queued

B

The response waits on the slowest of 50 parallel calls, so roughly 39% of requests touch at least one shard responding at or beyond its own p99

C

The coordinator queries shards one after another, so the per-shard latencies add up across the fan-out

D

Merging and re-ranking 50 partial result sets is CPU bound at the coordinator, and that merge step adds tens of milliseconds to every request it serves