4 / 6 Stream Processing, Windows & Watermarks

The Silent Partition

A Flink job reads one Kafka topic with 12 partitions and computes 5-minute event-time windows. Overnight, one partition receives no records for hours, and during that time the job stops emitting any window results at all, even though the other 11 partitions are busy. What is happening?

A

The idle partition's committed offset went stale and its source rewound to the earliest record, so the job is replaying old timestamps far behind the current windows

B

An operator's watermark is the minimum across its input channels, so the silent partition holds the whole job back; marking that source idle after a timeout lets the watermark advance

C

Watermarks are tracked per key rather than per partition, so keys with no overnight traffic never reach their window end; registering a keyed timer for each window forces the pending output

D

Backpressure from the 11 busy partitions is starving the window operator of processing slots, so raising the aggregation's parallelism restores the emitted output