Outside The Transaction

A read-process-write job runs one transactional producer per consumer instance. transactional.id is set, the consumer runs enable.auto.commit=false, and every downstream consumer runs isolation.level=read_committed.

producer.initTransactions();
while (true) {
  ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
  producer.beginTransaction();
  for (ConsumerRecord<String, String> r : records) {
    producer.send(new ProducerRecord<>(OUT, r.key(), transform(r.value())));
  }
  producer.commitTransaction();
  consumer.commitSync();
}

The pod is killed in the gap between commitTransaction() and commitSync(). What does a downstream read_committed consumer see once the replacement instance takes over the partition?

A

Nothing extra. commitTransaction() writes the consumer's position to __consumer_offsets as part of the transaction, and the trailing commitSync() is a redundant second write that only matters for lag reporting

B

Every output record of that batch twice: the transaction committed so the first copies are visible, and the input offsets never moved, so the new instance reads the same records and produces the outputs again inside a second committed transaction

C

Nothing extra. The producer is idempotent, so the broker recognises the resent records by their sequence numbers and drops the second copy before it is ever written

D

Nothing extra, and the batch is instead lost: the coordinator fences the old transactional.id when the new instance calls initTransactions, which aborts the transaction the dead pod had committed