Never A Dead Letter

This worker has run in production for a year. The queue has a redrive policy with maxReceiveCount of 5 and a dead-letter queue. The error log shows about 4,000 render failed lines a day. The dead-letter queue has never held a single message.

while True:
    resp = sqs.receive_message(QueueUrl=Q, MaxNumberOfMessages=10, WaitTimeSeconds=20)
    for m in resp.get("Messages", []):
        try:
            render(m["Body"])
        except Exception:
            log.exception("render failed")
        finally:
            sqs.delete_message(QueueUrl=Q, ReceiptHandle=m["ReceiptHandle"])

Why is the dead-letter queue empty?

A

maxReceiveCount of 5 means five failures inside one visibility window, and these failures are spread across separate receives

B

The finally block deletes the message on the failure path too, so no message is ever received a second time, the receive count never reaches 5, and roughly 4,000 jobs a day are silently thrown away

C

log.exception handles the exception, so boto3 reports the receive as successful and SQS never increments the receive count for that message

D

The redrive policy only applies to messages whose visibility timeout expires while in flight, and this worker always finishes inside the timeout