Six Limiters Not One

This module ships inside a service the runbook describes as stateless. It is imported once per process and the service runs on 6 replicas behind a round-robin load balancer.

# limiter.py
import time

RATE = 100.0        # tokens added per second
CAPACITY = 100.0
_BUCKET = {"tokens": CAPACITY, "last": time.monotonic()}

def allow() -> bool:
    now = time.monotonic()
    _BUCKET["tokens"] = min(CAPACITY, _BUCKET["tokens"] + (now - _BUCKET["last"]) * RATE)
    _BUCKET["last"] = now
    if _BUCKET["tokens"] >= 1.0:
        _BUCKET["tokens"] -= 1.0
        return True
    return False

A client hammers the endpoint from many connections. What rate does the downstream backend actually see?