Throttling Yourself First

A client library implements adaptive throttling. Over a rolling two-minute window it counts requests (calls the application attempted, including ones the library rejected locally) and accepts (calls the backend actually served). It drops new calls locally with probability

$p = \max\left(0,\ \frac{\text{requests} - K\cdot\text{accepts}}{\text{requests} + 1}\right)$

Right now requests is 1,800, accepts is 300, and $K = 2$ . What is $p$ , and what does it accomplish?

Zero. accepts is non-zero, so the backend is still serving traffic and the throttle only engages once it rejects everything.

About 0.67, so roughly two of every three new calls fail inside the client without a packet leaving it. The backend stops spending CPU on requests it was going to reject anyway, which is where a quota-exhausted service usually falls over.

About 0.67, but because requests counts locally rejected calls too, $p$ climbs toward 1 and the client stops calling the backend until an operator resets its counters.

About 0.83, from $(1800-300)/1801$ : $K$ scales the result rather than the accept count.