Question 1445

One Window Three Firings

An event-time job uses 5 minute tumbling windows, allowedLateness of 60 seconds, and a side output for dropped-late records. Consider the window W = [12:00:00, 12:05:00) and these five records, with the watermark value observed at the moment each one is handled:

record event time arrival watermark at arrival
r1 12:04:10 12:04:22 12:03:50
r2 12:04:55 12:05:08 12:04:36
r3 12:03:30 12:05:35 12:05:12
r4 12:04:40 12:06:15 12:05:47
r5 12:02:05 12:07:10 12:06:44

How many times does W emit, and which records reach the side output?

Window 12:00 to 12:05 fires when the watermark crosses 12:05:00 and is purged when it crosses 12:06:00 12:00:00 window [12:00, 12:05) 12:05:00 window end watermark crosses, window fires 12:06:00 state purged, later records go to the side output
A

W emits twice, once when the watermark passes 12:05:00 and once when it passes 12:06:00 with r3 and r4 folded into a single final result, and only r5 is dropped

B

W emits three times and only r5 is dropped: it fires when the watermark passes 12:05:00 with r1 and r2, then again for r3 and again for r4, both of which arrive while the watermark is still below 12:06:00

C

W emits once, with r1 and r2, and r3, r4 and r5 all reach the side output, because the watermark passing the window end is the cutoff for membership

D

W emits once, at the moment the watermark passes 12:06:00, containing r1 through r4, and only r5 is dropped, because allowed lateness delays the firing rather than retaining the state