Design a URL Shortener: System Design Interview Guide

99.99% on the redirect path, about 4 minutes per rolling 28-day window. Creation gets a looser target: 99.9%.

Two reasons for the gap. Failure costs more on reads: a broken redirect hits someone who isn’t your customer, with no context and no retry beyond a refresh, and they blame the destination brand. A clean creation failure surfaces to someone sitting in front of a form. Reliability costs less on reads: a redirect is a stateless lookup any replica can serve, while a write isn’t done until it’s durably stored.

This distinction matters for the architecture. It means the read path needs more redundancy than the write path. Multiple read replicas, a warm cache, and a load balancer with fast health checks are all direct responses to this requirement. The write path, which at 11.5 QPS is barely a whisper, can tolerate a brief primary failover without meaningful user impact.

10 ms is aggressive. A round-trip to a database in the same data centre is typically 1–5 ms under ideal conditions, but p99 latency accounts for the worst 1% of requests: slow queries, lock contention, garbage collection pauses. Hitting the DB on every redirect makes it very hard to stay under 10 ms at the 99th percentile.

Redis, by contrast, responds in under 1 ms for a simple key lookup. That headroom (1 ms for the cache lookup, plus network overhead) is why the <10 ms target effectively mandates a cache. Without it, the target is aspirational rather than achievable.

The <50 ms target on a cache miss is more relaxed by design. A miss means a DB read, which is slower but still expected to be rare (80%+ of requests should hit the cache). Setting a separate, looser target for misses is honest engineering, you're acknowledging the two paths have different cost profiles rather than pretending they're the same.

A lost URL mapping is catastrophic in a way that a lost click event is not. If short.ly/abc123 disappears from the database, every future user who clicks that link gets a 404. There is no recovery. The original long URL is gone. This is why URL mappings must be stored in a durable, replicated database with write-ahead logging, not in Redis (which is a cache, not a database, despite being able to persist data).

The framing "no silent loss" is deliberate. It means a write must not return success unless the data is safely persisted. An acknowledged write that later disappears due to async replication lag on a primary failure is the failure mode to design against. This points toward synchronous replication or quorum writes for the URL mapping table.

This is an explicit relaxation, and stating it out loud is just as important as stating the strict requirements. Not every part of the system needs the same guarantees. If a dashboard shows 9,842 clicks instead of 9,847 because five recent events are still in the Kafka pipeline, no user is harmed. The number will catch up within seconds.

This relaxation is what makes the async analytics architecture acceptable. If click counts needed to be exact and real-time, you'd need synchronous writes on the redirect path, which would blow the latency budget. By establishing eventual consistency as acceptable here, you unlock the entire async pipeline design: Kafka → Flink → ClickHouse, with the redirect path never waiting for any of it.

The key skill is knowing which parts of a system can tolerate relaxed consistency. URL mappings cannot. Click counts can. Stating that distinction explicitly and tracing it back to user impact is what separates a senior answer from a junior one.

Treating scale as a fixed assumption rather than a range is a common mistake. 1M URLs/day and 100:1 is a reasonable starting anchor. It's the kind of number you'd get from a mid-size product. But the interviewer may nudge you toward 100M URLs/day, or toward a single viral event generating 50,000 QPS for 10 minutes. The architecture should be defensible across that range, not brittle at the edges.

The read:write ratio is the more structurally important number. At 10:1, you'd still care about performance but might question whether a cache layer is worth its complexity. At 100:1, the cache is obvious. At 1000:1 (a single viral URL), the cache becomes the only thing standing between you and a DB meltdown. Naming this ratio early anchors every caching discussion that follows.

A useful pattern: establish the baseline (1M/day, 100:1), then proactively mention how the design degrades and recovers at 10× scale. This shows you've thought beyond the happy path without needing the interviewer to push you there.

Start with what you can anchor to: how many new URLs are created per day? 1 million is a reasonable starting assumption for a mid-scale system. Converting to per-second is a one-step calculation: 1M ÷ 86,400 ≈ 11.5 writes/sec. That's modest. A single DB node handles this without breaking a sweat.

The more interesting number is reads: for every URL created, how many times is it clicked? For a general-purpose shortener, 100:1 is reasonable, so 11.5 × 100 ≈ 1,150 reads/sec on average. Traffic is never flat, so apply a 5× peak multiplier for spikes (a tweet goes viral, a campaign launches), putting peak reads at ~5,750/sec. The ratio itself varies enormously by use case — a marketing campaign URL can run 10,000:1.

Why this matters: at 100:1, this system is almost entirely reads. That single insight justifies the entire caching strategy. If reads were only 2:1, you'd think about the design very differently.

Each URL row needs: the short code (~8 bytes), the long URL (average ~200 bytes for a real URL, but allow 500 bytes including metadata columns). At 1M rows/day for 5 years: 1M × 365 × 5 = 1.8 billion rows, totalling roughly 900 GB.

900 GB fits comfortably on a single modern database server. This is an important realisation: you don't shard for storage capacity here, you shard for availability and read throughput. Sharding a database that doesn't need it for capacity adds operational complexity for no gain; knowing this lets you justify starting with a simpler single-primary and read replica setup.

The clicks table is a different story. At 100:1, you're storing 100M click events per day, about 36 billion rows per year at just the base URL creation rate. At even 100 bytes per click row, that's 3.6 TB/year. This is why clicks needs a separate, write-optimised store from day one of the scaled design.

For the write path: 11.5 writes/sec × 500 bytes ≈ 6 KB/sec inbound. Negligible. For the read path: a redirect response is tiny, just an HTTP 302 with a Location header, roughly 200 bytes. At 5,750 peak reads/sec that's about 1.1 MB/sec outbound. Again, not a concern.

Bandwidth becomes interesting only if you serve the destination page through the shortener (you don't), or if you add analytics pixel tracking with embedded images. For a pure redirect service, bandwidth is never the constraint. Saying so in an interview shows you've thought about it and ruled it out rather than just ignoring it.

URL access follows a power-law distribution. A small fraction of URLs receive the vast majority of traffic. Caching the hot 20% of the working set typically yields an 80%+ cache hit rate. The question is: what does 20% actually cost in RAM?

The daily active working set is roughly 1M new URLs × 5 years × 365 days = 1.8B total URLs, but on any given day only a slice of those are active. A reasonable model: cache the top 20% of daily active URLs. If 10M URLs are active on a given day, caching 2M × 500 bytes = 1 GB of Redis memory covers the hot tail. Even at 10× that scale, you're talking 10 GB. A single Redis node comfortably holds this.

Why this matters for the design: knowing the cache fits entirely in memory on one or two nodes means you can justify a simple Redis setup rather than immediately reaching for Redis Cluster. Scale the architecture to what the numbers actually demand.

Why a layer-7 over a layer-4 load balancer? An L7 (HTTP-aware) load balancer lets you route by URL path. The write path (POST /shorten) and read path (GET /:code) have completely different traffic patterns. Reads are 100× more frequent. L7 lets you scale them independently, routing them to separate server pools, without any client changes.

Why not DNS round-robin? DNS has no health-check awareness. A failed server receives traffic until the DNS TTL expires, which is potentially minutes. A proper load balancer detects unhealthy nodes within seconds and removes them. That responsiveness is essential for a 99.99% availability target.

Why stateless? A stateless server holds no session data. Every request carries everything needed to process it. This is the single architectural decision that makes horizontal autoscaling trivial: spin up more servers, point the load balancer at them, done. No sticky sessions, no session replication, no coordination overhead.

What state is explicitly excluded? No in-flight request state, no user sessions, no per-connection counters. The only "state" is the pre-fetched ID range in memory (counter-based encoding), which is safe to lose on restart, since the server simply fetches a new range from Zookeeper (a distributed coordination service for leader election and shared config).

Why is this the most important component? The power-law access pattern from Section 3 means caching the hot 20% yields 80%+ hit rates. Without cache, every redirect hits the DB; with it, most complete in ~1 ms with no DB I/O whatsoever.

Why not use Redis as the primary store? Redis is an in-memory store. Data loss on failure is possible without careful persistence configuration, and even then it lags behind a relational DB in durability guarantees. The database is the source of truth; the cache is a performance optimisation layered on top of it.

PostgreSQL vs Cassandra: when to switch? urls stays single-writer, but be precise about why. Generated codes cannot collide across regions — the Zookeeper-leased ID ranges above guarantee that two allocators never share a number, which is a second reason to prefer the counter over a hash. The genuine constraints are narrower: PostgreSQL has no first-class multi-master story, and custom aliases — where the user picks the string — are a real cross-region race and a CP problem whatever your ID scheme. At 11.5 writes/sec neither is worth solving, and one primary with regional read replicas is correct. Staying single-writer is also why the primary runs synchronous_commit against a standby rather than async-by-default streaming replication, trading commit latency (and a possible write stall on a lone standby) for durability. Cassandra or DynamoDB earns its place as the eventual replacement for PostgreSQL on urls itself, once a single primary can no longer absorb the write rate and you want automatic resharding rather than manual splits. It is not the answer for clicks — those never land in an OLTP store at all (Section 7). For interviews, PostgreSQL + read replicas is still the right starting answer for URL mappings.

Why not log clicks synchronously in the redirect handler? A redirect must complete in <10 ms. Even a fast synchronous DB write adds 5–15 ms on every redirect. At 1,000 redirects/sec that's significant load; at 50,000/sec after a viral link, it becomes a bottleneck that degrades the entire service for all users. Decoupling with Kafka means the redirect path never waits on analytics.

Why Kafka over a background thread writing to the DB? Kafka is a durable buffer. If the consumer falls behind during a traffic spike, events queue safely with no data loss and no backpressure on API servers. A background thread writing directly to the DB has no such buffer; if the DB slows down, the thread queue grows in the API server's memory until OOM.

GET requests are by definition idempotent and cacheable. Browsers, proxies, and CDNs feel free to cache them. A creation operation is neither. Submitting the same GET request twice could create two short codes (with Approach B). Using POST signals to the entire HTTP stack that this request has side effects and must not be cached or replayed automatically.

The response is 201 Created rather than 200 OK, a small detail that signals a resource was created and allows the Location header to point to the new resource if needed.

Five things to validate on every creation request, in order of importance:

URL validity. Reject malformed URLs that would never resolve. Check for valid scheme (http:// or https://), reject bare IP addresses if you want to block direct IP linking, enforce a maximum length (URLs over ~2,000 chars cause issues in some browsers).

SSRF / egress control. If anything in your stack ever fetches long_url server-side (preview generation, malware scan), block private, link-local, and loopback ranges plus the cloud metadata IP (169.254.169.254). A hostname check at submit time isn't enough — DNS rebinding lets a name resolve public during validation and private at fetch time — so enforce this at the resolver/fetch layer, not just on the string you were handed.

Custom alias availability and safety. Check against reserved words (api, health, admin, blog), profanity blocklist, and existing aliases. Return 409 Conflict if taken.

Rate limiting. Limit creation requests per API key and per IP. URL shorteners are frequently abused for spam and phishing; without rate limiting a single actor can generate millions of short codes.

Expiry sanity. If expires_at is provided, reject values in the past and values beyond a maximum retention window (e.g. 10 years).

This column carries the most important index in the system. Every redirect, the dominant operation, is a point lookup by short_code. A unique B-tree index on it gives O(log n) lookups in the worst case, but with a hot working set in the buffer pool, it is effectively O(1) in practice.

Why VARCHAR(8) not VARCHAR(255)? Index pages are fixed-size. Narrower columns pack more entries per page, which means more of the index fits in memory. For a table that could hold hundreds of millions of rows, this difference matters. 8 characters comfortably covers a 7-character Base62 code with one spare byte.

Why store both id and short_code? The id is the internal primary key used for joins (e.g., clicks.url_id). The short_code is the external lookup key. Keeping them separate means your internal row identity is decoupled from the URL encoding scheme. If you ever change encoding strategies, you don't need to rewrite foreign keys across the database.

Most URLs never expire, so the column is nullable. A NULL value means "no expiry". This avoids storing a sentinel value like 9999-12-31 which looks awkward and can confuse application code.

Do not index this column on the main table. Expiry sweeps are a background job running infrequently, they don't need sub-millisecond performance. Adding an index on expires_at would slow down every insert for negligible gain. If sweep performance becomes a concern, run the job against a replica, or maintain a separate lightweight table of (short_code, expires_at) that is cheap to scan.

How does this interact with the cache? When a URL is created with an expiry, the cache TTL should be set to min(24h, expires_at − now). This ensures the cache entry naturally falls away when the URL expires, without needing a proactive invalidation sweep.

Strictly speaking, this is redundant. You can derive the total click count with SELECT COUNT(*) FROM clicks WHERE url_id = ?. But at scale that query becomes expensive if the clicks table has billions of rows. A denormalised counter on the urls row enables instant display of a click count in the UI.

Why not update it synchronously on every redirect? This is the counter hotspot problem. If a URL receives 10,000 clicks per second, you'd have 10,000 concurrent UPDATE urls SET click_count = click_count + 1 statements fighting for a row lock. The right approach is to update it periodically via the analytics pipeline, batch-aggregate click events in Kafka, then apply a single UPDATE once per minute per URL. Eventual consistency is acceptable here; users don't need sub-second click count precision.

clicks is append-only and grows without bound. A URL shortener with 1M new URLs/day and a 100:1 read:write ratio generates 100M click events per day (about 1,000 rows per second, continuously). Within a year that's 36 billion rows. That volume is incompatible with a traditional RDBMS table sitting alongside the operationally critical urls table.

The solution: separate the store at the infrastructure level. At L4, putting clicks in a separate table in the same DB is acceptable, and you should say so. At L5 and above, clicks stops being a table you write to at all: the redirect publishes the event to Kafka and returns, Flink aggregates it, ClickHouse serves the dashboard, and S3 Parquet holds the raw archive partitioned by date for ad hoc queries and backfill (Section 9 walks the pipeline stage by stage). A wide-column store like Cassandra or Bigtable is a defensible answer for raw clicks in isolation, but once the Kafka pipeline exists it is a third store with no query of its own — so pick one and say which.

Why store ip_hash not ip_address? A salted hash is pseudonymised, not anonymised — GDPR (Recital 26) still treats it as personal data, since whoever holds the salt can re-link it, and IPv4's ~4.3B addresses are brute-forceable against a known salt anyway. What hashing actually buys you is reduced blast radius if the store leaks and a cleaner data-minimisation story, not an exemption from scope — a rotating salt and short retention window do the real work. Say this precisely; claiming the hash "isn't personal data" is the kind of overclaim that costs points at L6+.

The dominant analytics query is: "give me all clicks for URL X between time A and time B." A composite index (url_id, clicked_at) handles this perfectly. The leftmost column (url_id) filters down to a specific URL's rows, and the rightmost column (clicked_at) supports efficient range scans within that set.

Why not (clicked_at, url_id)? Reversing the order would make the index useful for "give me all clicks across all URLs in a time range", a much less common query. Always index in the order that matches your most frequent access pattern: filter first, range second.

At large scale, this index itself becomes enormous — and it stops being the thing that answers the query. Once clicks move to the pipeline, the same access pattern is served by a ClickHouse MergeTree with ORDER BY (url_id, clicked_at): the sort key does the work the composite index did, and the columnar layout means a per-URL time-range scan reads only the columns the dashboard asked for rather than whole rows.

Redis handles most traffic well, but a single viral URL can saturate the network connection to the specific Redis shard holding that key, even if Redis itself is healthy. The L1 local cache on each API server handles this case: requests served from L1 never touch the network at all. With a 1-second TTL, each API server effectively rate-limits its own outbound Redis traffic to at most one request per key per second, regardless of incoming load.

The two layers also fail independently. If Redis goes down, L1 continues serving hot entries for up to 1 second, during which a circuit breaker (trips after a failure threshold, then fails fast) can activate and route misses directly to the DB rather than hammering a failing Redis. Without L1, a Redis failure immediately becomes a DB flood.

LRU eviction handles memory pressure: when the cache fills up, the least recently used entries are dropped to make room. TTL handles correctness: entries that haven't been evicted are still expired after 24 hours, preventing perpetually stale data from accumulating for obscure URLs that were cached long ago and never requested since.

They solve different problems and you need both. LRU alone means a long-tail URL cached six months ago stays forever, even if the underlying long URL was changed. TTL alone means a popular URL that's accessed every second still gets evicted after 24 hours and causes an unnecessary DB read.

For URLs with an expiry date, override the TTL with the min(24h, expires_at − now) formula from Section 7, so the cache entry never outlives the URL itself.

URL updates and deletions are uncommon; users rarely change a short URL once created. But when they do happen, the cache must be invalidated immediately or you risk serving a stale (or deleted) URL for up to the TTL duration. The correct approach is synchronous invalidation: on any write to the urls table, issue a DEL short_code to Redis as part of the same operation (not as a background job). If the DEL fails, the write should still succeed (the TTL will eventually clean it up), but log the failure and alert.

This is a case where cache invalidation is simpler than it looks. Because short_code is the exact cache key, there's no partial invalidation, no cache stampede (concurrent misses on one hot key all hitting the DB at once), and no need for versioning. Delete the key; the next request repopulates it from the DB.

As established in capacity estimation, the urls table at moderate scale (~900 GB) fits on a single node and doesn't need sharding for capacity. You shard when read throughput exceeds what a single primary + replica set can serve, or when you need geographic distribution for latency.

The shard key is short_code, using consistent hashing. The reasoning is direct: every redirect is a point lookup by short_code, so sharding on it ensures each request goes to exactly one shard, with no cross-shard queries and no scatter-gather aggregation (fanning a query out to every shard and merging results). The alternative (sharding on user_id) would be wrong: it optimises for the management UI (low traffic) at the expense of the redirect path (the dominant operation).

A global auto-incrementing counter is the simplest ID generator but a single point of contention at scale. Two patterns eliminate that contention:

Zookeeper range allocation: each API server pre-fetches a range of IDs (e.g. 1,000–2,000) from Zookeeper at startup. It burns through them locally with no network calls. When exhausted, it fetches the next range. Zookeeper sees only one request per thousand URLs created, effectively removing it from the hot path. The risk: if a server crashes mid-range, those IDs are lost. Given the key space is 3.5 trillion, a few thousand lost IDs is irrelevant.

Snowflake IDs: a 64-bit value composed of an unused sign bit (1), a timestamp (41 bits), datacenter ID (5 bits), machine ID (5 bits), and per-machine sequence (12 bits) — 1+41+5+5+12 = 64. Entirely self-contained, requiring no coordination or single point of contention. The tradeoff is clock skew: if a machine's clock jumps backward, ID monotonicity breaks. Handle with a clock guard that refuses to generate IDs if the system clock regresses.

The stand, stated once. Click events never enter an OLTP store. The redirect publishes to Kafka and returns; everything downstream is a consumer. Cassandra and DynamoDB appear in this design only as the large-scale alternative to PostgreSQL for urls, never for clicks. Quietly mixing the two is the most common way this answer loses coherence.

The pipeline is Kafka → Flink/Spark → ClickHouse → S3, and each stage exists for a specific reason. Kafka is the durable buffer: it decouples the API servers (producers) from the analytics processors (consumers), absorbs traffic spikes without backpressure, and retains events for 7 days allowing replay if a consumer has a bug. Flink or Spark Streaming aggregates raw click events into per-URL, per-hour, per-country counts, which is the shape your analytics dashboard actually queries. ClickHouse stores the aggregated metrics and answers dashboard queries in milliseconds due to its columnar storage format. S3 Parquet holds the raw events, partitioned by date, as the long-term archive for ad hoc queries and backfill.

Why not write directly from Kafka to ClickHouse? You could, but the aggregation step in Flink is important: it reduces 100M raw click rows/day to a far smaller set of pre-computed aggregates, keeping ClickHouse fast and cheap. Without it, every dashboard query would scan billions of rows.

GeoDNS routes users to the nearest regional cluster, reducing redirect latency from ~100 ms (cross-continent) to ~5–10 ms (local region). Each region gets its own Redis cache and a read replica of the DB. URL creation writes always go to the global primary, and it is the API tier that routes them there — a regional API server holds two connections, a local one for reads and one to the primary for writes. The replica never forwards anything; it accepts no writes at all. Say this explicitly, because "writes go to the primary" leaves open the wrong implementation, where the regional database is expected to proxy them. active-active writes are off the table for the urls table, to avoid split-brain (both regions accepting conflicting writes), since two regions could otherwise mint the same short_code for different URLs. Click events have no such conflict: each region publishes to its own local Kafka cluster and the aggregates merge downstream, so the analytics path really is active-active even though the write path is not.

"Async replication" and "durable" are answers to different questions. The cross-region link is asynchronous, but nothing about the commit depends on it. A create is durable the moment the primary and its synchronous standby — same region, different availability zone — have both flushed the WAL record, and that happens before EU-West hears about the row at all. The async replica buys read latency and a warm disaster-recovery target; it buys no durability, and it was never in the commit path. What it does cost you is RPO on losing the entire home region: writes committed but not yet shipped are gone, bounded by the replication lag. Continuous WAL archiving to object storage in a third region is the backstop that shrinks that window to the archive interval instead of the lag (see the durability deep-dive below). Making EU-West synchronous instead would put a cross-ocean flush — 80–150 ms — on every create and make EU an availability dependency of the write path: if EU became unreachable, writes on the primary would stall. For a shortener that is a bad trade in both directions.

The failure mode to reason through: a URL is created in region A, and a user in region B clicks it 200 ms later, before the async replication has propagated the new row to region B's replica. The correct behaviour is not a 404 but a fallback: on a cache miss that also misses the local replica, the read path falls through to the global primary. This adds latency for that one request but guarantees correctness. P99 replication lag between regions is typically under 100 ms, so this fallback is rare in practice.

Section 2 asserted that a lost URL mapping is unrecoverable while a lost click event is not. That asymmetry is a requirement, not a mechanism. Here is the mechanism, and it is four layers deep — candidates who say "we use a database, databases are durable" stall at L4.

1 — The commit. By default PostgreSQL acknowledges a commit once the WAL record is on the primary's disk. That is durable against a process crash but not against losing the node. Setting synchronous_commit = on with synchronous_standby_names = 'ANY 1 (s1, s2)' holds the acknowledgement until the record has also reached a standby. Name at least two candidate standbys: with a single named standby, losing it stalls every write on the primary, which converts a redundancy outage into a total one.

2 — The write-ahead log. fsync = on and full_page_writes = on are what make the WAL meaningful. Turning either off to buy write throughput means a power loss leaves a torn page and an unrecoverable cluster. This also assumes the storage layer honours fsync rather than acknowledging from a volatile write cache — worth one sentence in an interview, because it is the layer most people never check.

3 — Point-in-time recovery. Replication is not backup. A bad migration or an accidental DELETE replicates to every standby in milliseconds, perfectly. Continuous WAL archiving plus periodic base backups gives you restore-to-timestamp, which is the only defence against logical corruption. Keep the archive in a different failure domain from the cluster — a different account or project, not just a different bucket.

4 — Proof. An unrestored backup is a hypothesis. Run an automated weekly restore into a scratch cluster and assert a row count and a few known short_code lookups. The restore drill is also how you learn your real RTO, which is almost never the number in the design doc.

Spend the budget where it matters. The rest of the system is deliberately less durable, and you should say so out loud. Redis is a cache with no system-of-record role — losing it costs latency, not data. Kafka gets acks=all, replication.factor=3, min.insync.replicas=2, but the redirect path publishes fire-and-forget, so a click event can genuinely be lost in a broker-side outage; that is the relaxation Section 2 signed up for. S3 holds the raw archive with versioning enabled. Four different durability postures, each argued from the requirement rather than applied uniformly, is what the question is actually testing.