Upload And Transcode
Design the upload and processing pipeline behind a YouTube-style video platform: a creator uploads a file from a phone or a browser, and minutes later it is playable worldwide at multiple qualities.
Requirements
Functional:
- Resumable uploads of files up to 20 GB that survive a dropped connection, an app restart, or a switch from wifi to cellular.
- Transcode every source into an adaptive bitrate ladder (144p through 2160p) plus an audio-only track.
- Generate thumbnail candidates and a storyboard sprite sheet for scrubbing.
- Run a moderation check before a video can become publicly playable.
- The creator sees upload and processing progress, then publishes now or schedules.
Non-functional:
- p95 from upload complete to first playable rendition under 3 minutes for a 10 minute source, full ladder under 15 minutes.
- No duplicate publishes and no half-published videos, ever.
- Processing and storage cost per uploaded minute is a first class constraint, not an afterthought.
Scale
- 40M DAU, 500k uploads per day, peak hour runs 3x the daily average rate.
- Median source 600 MB, p95 2 GB, hard cap 20 GB. Average duration 8 minutes.
- About 300 TB of source bytes ingested per day.
- Playback traffic is roughly 200x upload traffic and is served from the edge.
- Upload status polling peaks around 20k QPS.
Out of scope
Player internals, the recommendation system, comments, live streaming, monetization, and the DRM license server.
Sign in to answer questions and track your progress
Sign In