Delivered Twice

Design the messaging backend for a WhatsApp-style app: 1:1 chat, group chat, delivery and read receipts, offline delivery, and end-to-end encryption.

Requirements

Functional:

• Text and media messages in 1:1 threads and groups of up to 1024 members.
• Per-recipient delivery and read receipts, aggregated per message for the sender.
• An offline recipient sees every message once after reconnecting, in a stable order, even if transport delivery is retried.
• A user may link up to 4 devices, and every device receives every message.
• End-to-end encrypted: the server never holds plaintext or content keys, so it cannot inspect, dedupe by content, or re-encrypt a message.

Non-functional:

• p50 send to delivered under 150 ms in-region, p99 under 500 ms.
• No visible duplicates and no silent loss.
• Losing an entire connection gateway node must not drop or reorder messages.

Scale

• 300M DAU, about 2 linked devices each, 60M concurrent connections at peak.
• 40B messages sent per day: roughly 460k/s average, 1.4M/s peak.
• Average text envelope 1 KB (ciphertext plus routing metadata). 5% of messages carry media averaging 200 KB.
• Average fanout is 3.5 recipient devices per message, so about 1.6M delivery events/s and roughly twice that again in receipt events.
• Media retained 14 days; undelivered messages retained 30 days, then dropped.

Out of scope

Voice and video calling, status and stories, payments, spam classification, encrypted cloud backup, and client UI.