6 / 6 Token Economics, Caching & Quotas

When 429s Arrive

Traffic is bursty and your service is taking 429 rate limit responses during peaks. Which measures genuinely help you stay under the provider's limits?

A

Lower max_tokens where possible, since the requested output budget is reserved against the token limit up front

B

Switch the calls to streaming, since tokens delivered incrementally are metered against a separate budget

C

Retry with exponential backoff and jitter, honoring the retry-after value instead of retrying immediately

D

Target steady-state usage well below the ceiling so bursts and retries still fit inside the limit

E

Raise client concurrency as soon as 429s appear, so more requests are in flight and the queued backlog clears faster