Foundations 0/8

Lesson 5 of 8

High-level design method

Work through a design answer from requirements to a request flow, then examine its capacity and failure cases.

1,718 words · 8 min read Useful background: How the web works, Back-of-envelope estimates

A high-level design explains how a request completes

High-level design explains how services and storage cooperate to complete a user operation under stated requirements. It assigns each component a responsibility, identifies the data crossing each boundary and states when an operation has succeeded. Following a request through the design should reveal where durable truth lives and what a caller observes when a required component is unavailable.

Functional requirements define supported operations, such as reading recent activity or creating a post. Quality requirements constrain their behavior, such as peak request rate, acceptable latency and tolerated data loss. “Show recent activity within 200 ms at 1,000 reads per second” can test a design, once its measurement population is defined. “Make it scalable” leaves both the workload and the success criterion unspecified.

Begin with the simplest complete request path. Define the API and the stored state it changes, then identify each dependency between request and response. A cache needs a key and a freshness rule. A queue needs a durable enqueue point and enough consumers. Adding either component changes the failure cases the design must explain.

Capacity reasoning follows the work through this path. The bottleneck is the stage whose available capacity first limits the workload being considered. Reducing work at that stage can expose another limit, so each proposed improvement needs a recalculation.

Give each stage a decision to produce

A sequence for a design interview or an architecture review
StageQuestionUseful output
ScopeWho reads or writes what, and what can wait?Two or three core operations and explicit exclusions.
TargetsWhat peak rate and user-visible guarantees matter?Traffic estimate, latency goal and data-loss tolerance.
Contracts and dataWhat identifies an operation and where is truth stored?Request/response examples and a small data model.
Request flowWhich component performs each operation?A path from caller through durable state to response.
Focused deep diveWhich limit appears first at the target load?A calculation and a change addressing that limit.
Failure and recoveryWhat happens after a timeout, crash or stale read?Retry ownership, degraded behavior and recovery plan.

Spend more time where the requirements create risk. A chat system may need a detailed reconnect and ordering discussion. A read-heavy catalog may need cache freshness and hot-key behavior. Equal time for every technology leaves the difficult requirement unexplained.

A complete answer for a URL shortener

Here is the whole sequence on one prompt, compressed to its decisions. The numbers continue the opening worked through in the design interview formats lesson.

  1. Requirements. Create a short link for a long URL, optionally with a custom alias, and redirect anyone who opens it. Analytics are out of scope for now. Redirects should have a server-side p99 under 100 ms, a code must never resolve to the wrong destination, and links do not expire.
  2. Estimates. 100 million new links a month is about 40 writes a second. With 100 redirects per link, reads average about 4,000 a second, and a fivefold peak gives 20,000. At 500 bytes per record, storage grows about 600 GB a year. The read rate, not the storage, shapes this design.
  3. API. Two operations, shown in the exchange below.
  4. Data model. One table keyed by code, holding the long URL, an owner id and the creation time. The primary key on code doubles as the uniqueness check.
  5. Request flow. To create, an application server generates a code and inserts the row; a duplicate-key error means the code is taken, so it generates another and retries. To redirect, the server looks the code up in a cache, falls back to the database on a miss and returns a 302 with the destination.
  6. Deep dive: generating codes. Seven base62 characters give 62^7, about 3.5 trillion codes. Random codes need no coordination between servers; after ten years at 1.2 billion links a year, about 0.3% of new codes would collide with an existing one and need a retry. A global counter encoded in base62 never collides, but it needs a coordinated sequence and produces guessable codes that let anyone enumerate every link.
  7. Deep dive: the redirect path. At 20,000 reads a second, a cache in front of the database carries most of the load, since code-to-URL mappings are small and rarely change. A viral link makes one cache key hot on one cache node; replicate hot keys, or let a CDN cache the redirect for a few seconds at the cost of counting fewer clicks. If the cache fails, the database must absorb the full read rate or the service must shed load, so size the read replicas for that case.
POST /links HTTP/1.1Content-Type: application/json{"url": "https://example.com/articles/2026/a-very-long-path", "alias": null}HTTP/1.1 201 CreatedLocation: /links/x7Kp2QaContent-Type: application/json{"code": "x7Kp2Qa", "short_url": "https://sho.rt/x7Kp2Qa"}GET /x7Kp2Qa HTTP/1.1HTTP/1.1 302 FoundLocation: https://example.com/articles/2026/a-very-long-path
The two operations as HTTP exchanges

Each step produced something the next one used. The 100 to 1 read ratio justified the cache, the uniqueness constraint settled correctness before anyone discussed how codes are generated, and each deep dive followed the requirement most at risk.

Calculate load along the path before adding servers

The redirect path had one obvious limit. Most designs have several stages in series, and the useful skill is finding which one limits the whole path. Consider an activity page receiving 1,000 reads per second. Each request reaches an application instance, reads an activity list and records a read notification. Assume capacities of 400 requests per second per application instance, 300 list reads per second for the database and 600 events per second for notification delivery.

With two application instances, the application tier could accept 800 requests per second. The database still performs one read for every request and can serve only 300, so it limits the complete synchronous path. Adding application instances increases offered work to that same database without increasing successful page capacity.

Now let a cache satisfy 75% of list reads. Each page request creates 0.25 database reads on average, so the database’s 300-read limit supports 300 ÷ 0.25 = 1,200 page requests per second under this hit rate. At the target rate, it receives 250 reads per second. The synchronous notification stage, at 600 events per second, becomes the lower limit.

A durable queue can move notification delivery after the page acknowledgment. With three application instances and the stated hit rate, the reply path can handle 1,000 requests per second. The queue still receives 1,000 events per second and its consumers still complete only 600. Backlog grows by 400 events per second, or 24,000 per minute. To sustain that rate, increase consumer throughput or reduce the events the product requires; a larger queue merely postpones exhaustion.

This change is acceptable only if the user operation may complete before notification delivery. The application must durably enqueue before acknowledging that promise, and workers need retry and duplicate-handling rules. If the notification is itself part of the required immediate result, moving it behind the acknowledgment would weaken the operation’s contract.

For a failure review, remove one application instance from the three-instance design. Its capacity becomes 800 requests per second, below the target, which gives you a concrete follow-up: provision spare capacity, reject lower-priority reads or serve an acceptable stale page. An autoscaling promise is incomplete unless it accounts for the time required to start usable capacity. Near saturation, queueing makes latency grow faster than load, so load-test the result rather than trusting the arithmetic alone.

Move the bottleneck through a request path

Which stage limits the page at 1,000 requests per second, and what does each fix expose next? Add application instances, raise the cache hit rate and move notifications off the reply path, in different orders.

  1. Application800 requests/s
  2. Cache hits bypass database0%
  3. Database1000 reads/s offered; limit 300
  4. NotificationsReply path: 600 events/s

Every read uses the app. Cache misses use the database. Every completed read emits one notification.

Reply-path bottleneck
Database
Served requests
300 requests/s
Mean service time, no queueing
108.0 ms
Notification queue growth
0 events/s

Incoming traffic exceeds reply capacity. Adding capacity to another stage does not remove this limit.

Synthetic capacities: 400 requests/s per app, 300 database reads/s and 600 notifications/s. Service times exclude contention and queueing. Synthetic times: app 8 ms, database miss 30 ms, durable enqueue 2 ms, synchronous notification 70 ms. At 75% hits with async delivery: 8 + 0.25 × 30 + 2 = 17.5 ms. Network transit and queue waiting are excluded.

Caching assumes activity lists can be stale for a stated interval and that the measured hit rate holds during the peak. A cold cache or an invalidation burst can send the full read rate back to the database, so reserve capacity or define degraded behavior for that case.

What stays true

Every admitted request contributes work to each synchronous dependency it uses; moving work to a queue preserves that work.

With a cache, database load equals admitted requests multiplied by the miss fraction. For notifications, every admitted request creates one event. A design is stable over a sustained interval only when every required stage, including background consumers, can process its incoming work.

Distinguish reply capacity from sustainable operation

Costs introduced by each change
ChoiceBenefitCost or condition
More application replicasRaises aggregate application capacity.Adds operational cost; shared database and notification limits remain.
Cache with hit fraction hReduces database reads to qps × (1 - h).Adds memory, invalidation and stale-read policy; cold-cache traffic needs a plan.
Asynchronous notificationsRemoves notification service time from the reply.Requires durable enqueue, retry handling and enough consumer throughput.
Read replicasSpread reads across copies of the database.Replication lag can serve stale reads; writes still go to one primary.

Design answers with an unaccounted request

  1. Add application replicas for a database bottleneck.

    Where it breaks
    Six instances still forward 1,000 reads per second to a database that handles 300.
    The fix
    Calculate work at each dependency and reduce or distribute the limiting work.
  2. Call the system stable because the reply path handles the load.

    Where it breaks
    The page serves 1,000 requests per second while its 600-event consumer accumulates 24,000 events per minute.
    The fix
    Calculate background arrival and service rates and define a backlog or age limit.
  3. Multiply average traffic by an unexplained number.

    Where it breaks
    A launch produces 2,000 reads per second while the design was sized for an assumed peak of 1,000.
    The fix
    State the source of the peak estimate and test how the design behaves above it.
  4. Use a cache hit rate as a failure guarantee.

    Where it breaks
    A cache restart changes a 75% hit rate to zero and overloads the database.
    The fix
    Describe cache warming, admission limits or stale fallback during recovery.

Deepen the decision that limits your design

Related topics

Further reading