Outside The Transaction
A read-process-write job runs one transactional producer per consumer instance. transactional.id is set, the consumer runs enable.auto.commit=false, and every downstream consumer runs isolation.level=read_committed.
producer.initTransactions();
while (true) {
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
producer.beginTransaction();
for (ConsumerRecord<String, String> r : records) {
producer.send(new ProducerRecord<>(OUT, r.key(), transform(r.value())));
}
producer.commitTransaction();
consumer.commitSync();
}
The pod is killed in the gap between commitTransaction() and commitSync(). What does a downstream read_committed consumer see once the replacement instance takes over the partition?
Sign in to answer questions and track your progress
Sign In