A high-level design explains how a request completes
High-level design explains how services and storage cooperate to complete a user operation under stated requirements. It assigns each component a responsibility, identifies the data crossing each boundary and states when an operation has succeeded. Following a request through the design should reveal where durable truth lives and what a caller observes when a required component is unavailable.
Functional requirements define supported operations, such as reading recent activity or creating a post. Quality requirements constrain their behavior, such as peak request rate, acceptable latency and tolerated data loss. “Show recent activity within 200 ms at 1,000 reads per second” can test a design, once its measurement population is defined. “Make it scalable” leaves both the workload and the success criterion unspecified.
Begin with the simplest complete request path. Define the API and the stored state it changes, then identify each dependency between request and response. A cache needs a key and a freshness rule. A queue needs a durable enqueue point and enough consumers. Adding either component changes the failure cases the design must explain.
Capacity reasoning follows the work through this path. The bottleneck is the stage whose available capacity first limits the workload being considered. Reducing work at that stage can expose another limit, so each proposed improvement needs a recalculation.
Give each stage a decision to produce
| Stage | Question | Useful output |
|---|---|---|
| Scope | Who reads or writes what, and what can wait? | Two or three core operations and explicit exclusions. |
| Targets | What peak rate and user-visible guarantees matter? | Traffic estimate, latency goal and data-loss tolerance. |
| Contracts and data | What identifies an operation and where is truth stored? | Request/response examples and a small data model. |
| Request flow | Which component performs each operation? | A path from caller through durable state to response. |
| Focused deep dive | Which limit appears first at the target load? | A calculation and a change addressing that limit. |
| Failure and recovery | What happens after a timeout, crash or stale read? | Retry ownership, degraded behavior and recovery plan. |
Spend more time where the requirements create risk. A chat system may need a detailed reconnect and ordering discussion. A read-heavy catalog may need cache freshness and hot-key behavior. Equal time for every technology leaves the difficult requirement unexplained.
A complete answer for a URL shortener
Here is the whole sequence on one prompt, compressed to its decisions. The numbers continue the opening worked through in the design interview formats lesson.
- Requirements. Create a short link for a long URL, optionally with a custom alias, and redirect anyone who opens it. Analytics are out of scope for now. Redirects should have a server-side p99 under 100 ms, a code must never resolve to the wrong destination, and links do not expire.
- Estimates. 100 million new links a month is about 40 writes a second. With 100 redirects per link, reads average about 4,000 a second, and a fivefold peak gives 20,000. At 500 bytes per record, storage grows about 600 GB a year. The read rate, not the storage, shapes this design.
- API. Two operations, shown in the exchange below.
- Data model. One table keyed by code, holding the long URL, an owner id and the creation time. The primary key on code doubles as the uniqueness check.
- Request flow. To create, an application server generates a code and inserts the row; a duplicate-key error means the code is taken, so it generates another and retries. To redirect, the server looks the code up in a cache, falls back to the database on a miss and returns a 302 with the destination.
- Deep dive: generating codes. Seven base62 characters give 62^7, about 3.5 trillion codes. Random codes need no coordination between servers; after ten years at 1.2 billion links a year, about 0.3% of new codes would collide with an existing one and need a retry. A global counter encoded in base62 never collides, but it needs a coordinated sequence and produces guessable codes that let anyone enumerate every link.
- Deep dive: the redirect path. At 20,000 reads a second, a cache in front of the database carries most of the load, since code-to-URL mappings are small and rarely change. A viral link makes one cache key hot on one cache node; replicate hot keys, or let a CDN cache the redirect for a few seconds at the cost of counting fewer clicks. If the cache fails, the database must absorb the full read rate or the service must shed load, so size the read replicas for that case.
POST /links HTTP/1.1Content-Type: application/json{"url": "https://example.com/articles/2026/a-very-long-path", "alias": null}HTTP/1.1 201 CreatedLocation: /links/x7Kp2QaContent-Type: application/json{"code": "x7Kp2Qa", "short_url": "https://sho.rt/x7Kp2Qa"}GET /x7Kp2Qa HTTP/1.1HTTP/1.1 302 FoundLocation: https://example.com/articles/2026/a-very-long-pathEach step produced something the next one used. The 100 to 1 read ratio justified the cache, the uniqueness constraint settled correctness before anyone discussed how codes are generated, and each deep dive followed the requirement most at risk.
Calculate load along the path before adding servers
The redirect path had one obvious limit. Most designs have several stages in series, and the useful skill is finding which one limits the whole path. Consider an activity page receiving 1,000 reads per second. Each request reaches an application instance, reads an activity list and records a read notification. Assume capacities of 400 requests per second per application instance, 300 list reads per second for the database and 600 events per second for notification delivery.
With two application instances, the application tier could accept 800 requests per second. The database still performs one read for every request and can serve only 300, so it limits the complete synchronous path. Adding application instances increases offered work to that same database without increasing successful page capacity.
Now let a cache satisfy 75% of list reads. Each page request creates 0.25 database reads on average, so the database’s 300-read limit supports 300 ÷ 0.25 = 1,200 page requests per second under this hit rate. At the target rate, it receives 250 reads per second. The synchronous notification stage, at 600 events per second, becomes the lower limit.
A durable queue can move notification delivery after the page acknowledgment. With three application instances and the stated hit rate, the reply path can handle 1,000 requests per second. The queue still receives 1,000 events per second and its consumers still complete only 600. Backlog grows by 400 events per second, or 24,000 per minute. To sustain that rate, increase consumer throughput or reduce the events the product requires; a larger queue merely postpones exhaustion.
This change is acceptable only if the user operation may complete before notification delivery. The application must durably enqueue before acknowledging that promise, and workers need retry and duplicate-handling rules. If the notification is itself part of the required immediate result, moving it behind the acknowledgment would weaken the operation’s contract.
For a failure review, remove one application instance from the three-instance design. Its capacity becomes 800 requests per second, below the target, which gives you a concrete follow-up: provision spare capacity, reject lower-priority reads or serve an acceptable stale page. An autoscaling promise is incomplete unless it accounts for the time required to start usable capacity. Near saturation, queueing makes latency grow faster than load, so load-test the result rather than trusting the arithmetic alone.
Move the bottleneck through a request path
Which stage limits the page at 1,000 requests per second, and what does each fix expose next? Add application instances, raise the cache hit rate and move notifications off the reply path, in different orders.
- Application800 requests/s
- Cache hits bypass database0%
- Database1000 reads/s offered; limit 300
- NotificationsReply path: 600 events/s
Every read uses the app. Cache misses use the database. Every completed read emits one notification.
- Reply-path bottleneck
- Database
- Served requests
- 300 requests/s
- Mean service time, no queueing
- 108.0 ms
- Notification queue growth
- 0 events/s
Incoming traffic exceeds reply capacity. Adding capacity to another stage does not remove this limit.
Synthetic capacities: 400 requests/s per app, 300 database reads/s and 600 notifications/s. Service times exclude contention and queueing. Synthetic times: app 8 ms, database miss 30 ms, durable enqueue 2 ms, synchronous notification 70 ms. At 75% hits with async delivery: 8 + 0.25 × 30 + 2 = 17.5 ms. Network transit and queue waiting are excluded.
Caching assumes activity lists can be stale for a stated interval and that the measured hit rate holds during the peak. A cold cache or an invalidation burst can send the full read rate back to the database, so reserve capacity or define degraded behavior for that case.
What stays true
Every admitted request contributes work to each synchronous dependency it uses; moving work to a queue preserves that work.
With a cache, database load equals admitted requests multiplied by the miss fraction. For notifications, every admitted request creates one event. A design is stable over a sustained interval only when every required stage, including background consumers, can process its incoming work.
Distinguish reply capacity from sustainable operation
| Choice | Benefit | Cost or condition |
|---|---|---|
| More application replicas | Raises aggregate application capacity. | Adds operational cost; shared database and notification limits remain. |
| Cache with hit fraction h | Reduces database reads to qps × (1 - h). | Adds memory, invalidation and stale-read policy; cold-cache traffic needs a plan. |
| Asynchronous notifications | Removes notification service time from the reply. | Requires durable enqueue, retry handling and enough consumer throughput. |
| Read replicas | Spread reads across copies of the database. | Replication lag can serve stale reads; writes still go to one primary. |
Design answers with an unaccounted request
Add application replicas for a database bottleneck.
- Where it breaks
- Six instances still forward 1,000 reads per second to a database that handles 300.
- The fix
- Calculate work at each dependency and reduce or distribute the limiting work.
Call the system stable because the reply path handles the load.
- Where it breaks
- The page serves 1,000 requests per second while its 600-event consumer accumulates 24,000 events per minute.
- The fix
- Calculate background arrival and service rates and define a backlog or age limit.
Multiply average traffic by an unexplained number.
- Where it breaks
- A launch produces 2,000 reads per second while the design was sized for an assumed peak of 1,000.
- The fix
- State the source of the peak estimate and test how the design behaves above it.
Use a cache hit rate as a failure guarantee.
- Where it breaks
- A cache restart changes a 75% hit rate to zero and overloads the database.
- The fix
- Describe cache warming, admission limits or stale fallback during recovery.
Deepen the decision that limits your design
Related topics
- Design interview formats →
What each round expects you to produce, and the first ten minutes of this answer.
- Back-of-envelope estimates →
Derive the traffic and storage numbers that feed a design.
- Caching strategies →
Explain freshness, invalidation and cache misses.
- Backpressure →
Control arrivals when downstream capacity is lower.
Further reading
- The System Design Primer A free, community-maintained index of topics and worked example designs. github.com (opens in a new tab)
- Google SRE: Handling Overload Production experience with overload, queue bounds and load shedding. sre.google (opens in a new tab)
- AWS Builders’ Library: Static stability Provision capacity that remains available during a failure, rather than relying on recovery-time changes. builder.aws.com (opens in a new tab)