Short Link Long Tail

Design a URL shortener: it turns long links into short codes and serves the redirect at very high volume.

Requirements

Functional:

  • Create a short code for a submitted URL and return the full short link.
  • Redirect a visitor from the short code to the destination.
  • Support user-chosen custom aliases in the same namespace as generated codes.
  • Support an optional expiry date, after which the link stops resolving.
  • Report per-link click analytics: total clicks, clicks over time, coarse geography, and referrer.

Non-functional:

  • Redirects are the product. They stay fast and available even while the create path or the analytics pipeline is degraded.
  • A code is never reused and never resolves to the wrong destination.
  • Analytics may lag by a minute. Redirects may not lag at all.

Scale

  • 5M new links per day, about 58 writes/sec average and 200/sec at peak.
  • 2B redirects per day, about 23k reads/sec average and 70k/sec at peak, so the read to write ratio is roughly 400:1.
  • Destination URLs average 120 bytes and are capped at 2KB.
  • Mapping rows grow about 0.9TB per year. Click events, stored raw, grow about 400GB per day.
  • Target p99 of 50ms for a redirect measured at the edge, and 200ms for link creation.
  • One link can take 30% of all traffic for several hours when it lands on a large social feed.

Out of scope

Account signup and billing, the dashboard front end, QR image rendering, and malware scanning beyond a blocklist lookup.