4 / 6 Timeouts, Retries & Circuit Breakers

Eight Percent Took Everything

A frontend has a fixed pool of 600 worker threads. Normal traffic is 400 requests per second with a mean service time of 40 ms, so it normally needs about 16 threads.

A dependency used by 8 percent of requests starts black-holing: those calls neither succeed nor fail, they sit until the client deadline of 60 seconds. Traffic does not change.

Roughly what share of all requests can the frontend still serve?

A

About 92 percent succeed and the rest are merely slower, because a full pool queues arrivals instead of rejecting them.

B

About 31 percent. By L=λWL = \lambda W the hung 8 percent alone demand 0.08×400×60=19200.08 \times 400 \times 60 = 1920 threads against a pool of 600, so they crowd out the healthy 92 percent, which needs only 0.92×400×0.04≈150.92 \times 400 \times 0.04 \approx 15.

C

About 92 percent. The hung calls are parked on network IO rather than burning CPU, so they do not hold a worker thread.

D

About 50 percent, because once both classes saturate the pool they end up sharing it evenly.