One Customer One Partition

A topic has 12 partitions and records are keyed by customer_id. One customer, an integration partner running a nightly bulk sync, produces about 40 percent of all records. Lag on partition 7 climbs into the millions every night while the other 11 partitions sit near zero. Consumers must apply each customer's events in order.

What actually helps?

Turn on compaction for the topic so the bulk sync's superseded updates for that customer are removed before the consumer has to read them

Give the hot customer its own topic or a composite key such as customer_id:bucket, and accept that events for that one customer are no longer totally ordered unless you sequence them downstream

Raise the partition count from 12 to 24 so the hash space is wider and the heavy key's share of any one partition is halved

Add consumers to the group so partition 7 is served by several members working through its backlog in parallel