System design 0/3

System design, Messaging and streams

What is backpressure?

Also called flow control

353 words · 2 min read

Backpressure is feedback that makes a producer reduce its output when a downstream consumer cannot keep up. A bounded buffer gives the consumer time to absorb a burst. When the buffer fills, the system must slow production, reject work or discard selected data. Those choices have different consequences for correctness.

A log shipper shows why the choice is necessary. The application generates 30,000 log lines a second, while the search cluster indexes 20,000. Unsent lines accumulate at 10,000 a second. At 500 bytes per line, their payload alone grows by 5 MB a second, before object overhead and the shipper’s other memory use.

A buffer stores the difference between arrival and service rates

The backlog changes by arrivals minus completed work. When the log rate exceeds indexing capacity by 10,000 lines a second, every extra second adds another 10,000 waiting lines. If the burst lasts 10 seconds and arrivals then fall to 10,000 a second, the consumer has 10,000 a second of spare capacity and clears that 100,000-line backlog in another 10 seconds.

Waiting time grows with the backlog. If 100,000 lines are already ahead of a newly accepted line and indexing continues at 20,000 a second, those preceding lines require about 5 seconds of service. This estimate assumes steady capacity and a first-in, first-out consumer. A queue can fit in memory while still violating a freshness target by minutes. Little's law states the same relation for any stable system: the average number of items waiting equals the arrival rate times the average time each spends inside, so at a steady 20,000 lines a second, a backlog that averages 100,000 lines means an average stay of 5 seconds.

Run a producer faster than its consumer

What happens to memory, waiting time and lost work when a producer offering 3 items a tick feeds a consumer that takes 2, through an 8-slot buffer? Run the same overload under each full-buffer policy, then change the two rates to find when the buffer stays empty.

Producer3 per tick

Buffer: 0 of 8 slots used, newest on the left

Consumer2 per tickDone: 0
Tick
0
Oldest waited
0
Produced
0
Dropped
0
Held back
0

When the consumer is at least as fast as the producer, every policy behaves the same, because the buffer never fills.

Backpressure Decide what happens when a producer outruns its consumer: buffer, block, drop or shed load. Read the full lesson