Design a URL Shortener: System Design Interview Guide

99.99% means roughly 52 minutes of downtime per year. The justification is asymmetric: a redirect failure is immediately visible to an end user, meaning they clicked a link and got nothing. A creation failure is far less damaging; users can retry, or the failure is invisible entirely (a background process that tries again).

This distinction matters for the architecture. It means the read path needs more redundancy than the write path. Multiple read replicas, a warm cache, and a load balancer with fast health checks are all direct responses to this requirement. The write path, which at 11.5 QPS is barely a whisper, can tolerate a brief primary failover without meaningful user impact.

10 ms is aggressive. A round-trip to a database in the same data centre is typically 1–5 ms under ideal conditions, but p99 latency accounts for the worst 1% of requests: slow queries, lock contention, garbage collection pauses. Hitting the DB on every redirect makes it very hard to stay under 10 ms at the 99th percentile.

Redis, by contrast, responds in under 1 ms for a simple key lookup. That headroom (1 ms for the cache lookup, plus network overhead) is why the <10 ms target effectively mandates a cache. Without it, the target is aspirational rather than achievable.

The <50 ms target on a cache miss is more relaxed by design. A miss means a DB read, which is slower but still expected to be rare (80%+ of requests should hit the cache). Setting a separate, looser target for misses is honest engineering, you're acknowledging the two paths have different cost profiles rather than pretending they're the same.

A lost URL mapping is catastrophic in a way that a lost click event is not. If short.ly/abc123 disappears from the database, every future user who clicks that link gets a 404. There is no recovery. The original long URL is gone. This is why URL mappings must be stored in a durable, replicated database with write-ahead logging, not in Redis (which is a cache, not a database, despite being able to persist data).

The framing "no silent loss" is deliberate. It means a write must not return success unless the data is safely persisted. An acknowledged write that later disappears due to async replication lag on a primary failure is the failure mode to design against. This points toward synchronous replication or quorum writes for the URL mapping table.

This is an explicit relaxation, and stating it out loud is just as important as stating the strict requirements. Not every part of the system needs the same guarantees. If a dashboard shows 9,842 clicks instead of 9,847 because five recent events are still in the Kafka pipeline, no user is harmed. The number will catch up within seconds.

This relaxation is what makes the async analytics architecture acceptable. If click counts needed to be exact and real-time, you'd need synchronous writes on the redirect path, which would blow the latency budget. By establishing eventual consistency as acceptable here, you unlock the entire async pipeline design: Kafka → Flink → ClickHouse, with the redirect path never waiting for any of it.

The key skill is knowing which parts of a system can tolerate relaxed consistency. URL mappings cannot. Click counts can. Stating that distinction explicitly and tracing it back to user impact is what separates a senior answer from a junior one.

Treating scale as a fixed assumption rather than a range is a common mistake. 1M URLs/day and 100:1 is a reasonable starting anchor. It's the kind of number you'd get from a mid-size product. But the interviewer may nudge you toward 100M URLs/day, or toward a single viral event generating 50,000 QPS for 10 minutes. The architecture should be defensible across that range, not brittle at the edges.

The read:write ratio is the more structurally important number. At 10:1, you'd still care about performance but might question whether a cache layer is worth its complexity. At 100:1, the cache is obvious. At 1000:1 (a single viral URL), the cache becomes the only thing standing between you and a DB meltdown. Naming this ratio early anchors every caching discussion that follows.

A useful pattern: establish the baseline (1M/day, 100:1), then proactively mention how the design degrades and recovers at 10× scale. This shows you've thought beyond the happy path without needing the interviewer to push you there.

Start with what you can anchor to: how many new URLs are created per day? 1 million is a reasonable starting assumption for a mid-scale system. Converting to per-second is a one-step calculation: 1M ÷ 86,400 ≈ 11.5 writes/sec. That's modest. A single DB node handles this without breaking a sweat.

The more interesting number is reads. Ask yourself: for every URL created, how many times is it clicked? For a general-purpose shortener, 100:1 is a reasonable ratio. A marketing campaign URL might be 10,000:1. That gives you 1,150 reads/sec on average, but traffic is never flat. Apply a 5× peak multiplier for spikes (a tweet goes viral, a campaign launches), which puts peak reads at ~5,750/sec.

Why this matters: at 100:1, this system is almost entirely reads. That single insight justifies the entire caching strategy. If reads were only 2:1, you'd think about the design very differently.

Each URL row needs: the short code (~8 bytes), the long URL (average ~200 bytes for a real URL, but allow 500 bytes including metadata columns). At 1M rows/day for 5 years: 1M × 365 × 5 = 1.8 billion rows, totalling roughly 900 GB.

900 GB fits comfortably on a single modern database server. This is an important realisation: you don't shard for storage capacity here, you shard for availability and read throughput. Sharding a database that doesn't need it for capacity adds operational complexity for no gain; knowing this lets you justify starting with a simpler single-primary and read replica setup.

The clicks table is a different story. At 100:1, you're storing 100M click events per day, about 36 billion rows per year at just the base URL creation rate. At even 100 bytes per click row, that's 3.6 TB/year. This is why clicks needs a separate, write-optimised store from day one of the scaled design.

For the write path: 11.5 writes/sec × 500 bytes ≈ 6 KB/sec inbound. Negligible. For the read path: a redirect response is tiny, just an HTTP 302 with a Location header, roughly 200 bytes. At 5,750 peak reads/sec that's about 1.1 MB/sec outbound. Again, not a concern.

Bandwidth becomes interesting only if you serve the destination page through the shortener (you don't), or if you add analytics pixel tracking with embedded images. For a pure redirect service, bandwidth is never the constraint. Saying so in an interview shows you've thought about it and ruled it out rather than just ignoring it.

URL access follows a power-law distribution. A small fraction of URLs receive the vast majority of traffic. Caching the hot 20% of the working set typically yields an 80%+ cache hit rate. The question is: what does 20% actually cost in RAM?

The daily active working set is roughly 1M new URLs × 5 years × 365 days = 1.8B total URLs, but on any given day only a slice of those are active. A reasonable model: cache the top 20% of daily active URLs. If 10M URLs are active on a given day, caching 2M × 500 bytes = 1 GB of Redis memory covers the hot tail. Even at 10× that scale, you're talking 10 GB. A single Redis node comfortably holds this.

Why this matters for the design: knowing the cache fits entirely in memory on one or two nodes means you can justify a simple Redis setup rather than immediately reaching for Redis Cluster. Scale the architecture to what the numbers actually demand.

Why L7 over L4? An L7 (HTTP-aware) load balancer lets you route by URL path. The write path (POST /shorten) and read path (GET /:code) have completely different traffic patterns. Reads are 100× more frequent. L7 lets you scale them independently, routing them to separate server pools, without any client changes.

Why not DNS round-robin? DNS has no health-check awareness. A failed server receives traffic until the DNS TTL expires, which is potentially minutes. A proper load balancer detects unhealthy nodes within seconds and removes them. That responsiveness is essential for a 99.99% availability target.

Why stateless? A stateless server holds no session data. Every request carries everything needed to process it. This is the single architectural decision that makes horizontal autoscaling trivial: spin up more servers, point the load balancer at them, done. No sticky sessions, no session replication, no coordination overhead.

What state is explicitly excluded? No in-flight request state, no user sessions, no per-connection counters. The only "state" is the pre-fetched ID range in memory (counter-based encoding), which is safe to lose on restart, since the server simply fetches a new range from Zookeeper.

Why is this the most important component? URL access follows a heavy power-law distribution, where a small fraction of URLs (viral links, popular campaigns) receives the vast majority of clicks. Caching 20% of the working set by count typically yields 80%+ hit rates. Without cache, every redirect hits the DB. With it, most complete in ~1 ms with no DB I/O whatsoever.

Why Redis over Memcached? Redis supports native TTL per key (critical for URL expiration), persistence options (RDB/AOF snapshots for warm-up after restart), and Cluster mode for horizontal sharding. Memcached is marginally faster for pure string caching but lacks all three of these operationally important features.

Why not use Redis as the primary store? Redis is an in-memory store. Data loss on failure is possible without careful persistence configuration, and even then it lags behind a relational DB in durability guarantees. The database is the source of truth; the cache is a performance optimisation layered on top of it.

MySQL vs Cassandra: when to switch? MySQL handles hundreds of millions of URL records comfortably on a single node with read replicas. Reach for Cassandra (or DynamoDB) when you need multi-region active-active writes, or when manual sharding in MySQL becomes operationally painful. For most interview scenarios, starting with MySQL + read replicas is correct, jumping straight to Cassandra invites difficult questions about tunable consistency.

Why not log clicks synchronously in the redirect handler? A redirect must complete in <10 ms. Even a fast synchronous DB write adds 5–15 ms on every redirect. At 1,000 redirects/sec that's significant load; at 50,000/sec after a viral link, it becomes a bottleneck that degrades the entire service for all users. Decoupling with Kafka means the redirect path never waits on analytics.

Why Kafka over a background thread writing to the DB? Kafka is a durable buffer. If the consumer falls behind during a traffic spike, events queue safely with no data loss and no backpressure on API servers. A background thread writing directly to the DB has no such buffer; if the DB slows down, the thread queue grows in the API server's memory until OOM.

GET requests are by definition idempotent and cacheable. Browsers, proxies, and CDNs feel free to cache them. A creation operation is neither. Submitting the same GET request twice could create two short codes (with Approach B). Using POST signals to the entire HTTP stack that this request has side effects and must not be cached or replayed automatically.

The response is 201 Created rather than 200 OK, a small detail that signals a resource was created and allows the Location header to point to the new resource if needed.

Four things to validate on every creation request, in order of importance:

URL validity. Reject malformed URLs that would never resolve. Check for valid scheme (http:// or https://), reject bare IP addresses if you want to block direct IP linking, enforce a maximum length (URLs over ~2,000 chars cause issues in some browsers).

Custom alias availability and safety. Check against reserved words (api, health, admin, blog), profanity blocklist, and existing aliases. Return 409 Conflict if taken.

Rate limiting. Limit creation requests per API key and per IP. URL shorteners are frequently abused for spam and phishing; without rate limiting a single actor can generate millions of short codes.

Expiry sanity. If expires_at is provided, reject values in the past and values beyond a maximum retention window (e.g. 10 years).

This column carries the most important index in the system. Every redirect, the dominant operation, is a point lookup by short_code. A unique B-tree index on it gives O(log n) lookups in the worst case, but with a hot working set in the buffer pool, it is effectively O(1) in practice.

Why VARCHAR(8) not VARCHAR(255)? Index pages are fixed-size. Narrower columns pack more entries per page, which means more of the index fits in memory. For a table that could hold hundreds of millions of rows, this difference matters. 8 characters comfortably covers a 7-character Base62 code with one spare byte.

Why store both id and short_code? The id is the internal primary key used for joins (e.g., clicks.url_id). The short_code is the external lookup key. Keeping them separate means your internal row identity is decoupled from the URL encoding scheme. If you ever change encoding strategies, you don't need to rewrite foreign keys across the database.

Most URLs never expire, so the column is nullable. A NULL value means "no expiry". This avoids storing a sentinel value like 9999-12-31 which looks awkward and can confuse application code.

Do not index this column on the main table. Expiry sweeps are a background job running infrequently, they don't need sub-millisecond performance. Adding an index on expires_at would slow down every insert for negligible gain. If sweep performance becomes a concern, run the job against a replica, or maintain a separate lightweight table of (short_code, expires_at) that is cheap to scan.

How does this interact with the cache? When a URL is created with an expiry, the cache TTL should be set to min(24h, expires_at − now). This ensures the cache entry naturally falls away when the URL expires, without needing a proactive invalidation sweep.

Strictly speaking, this is redundant. You can derive the total click count with SELECT COUNT(*) FROM clicks WHERE url_id = ?. But at scale that query becomes expensive if the clicks table has billions of rows. A denormalised counter on the urls row enables instant display of a click count in the UI.

Why not update it synchronously on every redirect? This is the counter hotspot problem. If a URL receives 10,000 clicks per second, you'd have 10,000 concurrent UPDATE urls SET click_count = click_count + 1 statements fighting for a row lock. The right approach is to update it periodically via the analytics pipeline, batch-aggregate click events in Kafka, then apply a single UPDATE once per minute per URL. Eventual consistency is acceptable here; users don't need sub-second click count precision.

clicks is append-only and grows without bound. A URL shortener with 1M new URLs/day and a 100:1 read:write ratio generates 100M click events per day (about 1,000 rows per second, continuously). Within a year that's 36 billion rows. That volume is incompatible with a traditional RDBMS table sitting alongside the operationally critical urls table.

The solution: separate the store at the infrastructure level. At L4, putting clicks in a separate table in the same DB is acceptable. At L5/L6, it belongs in a write-optimised wide-column store (Cassandra, Bigtable) or streamed directly to a columnar analytics store (ClickHouse). At L7/L8, raw click events live in S3 Parquet partitioned by date, queryable via Athena or BigQuery, with ClickHouse serving as the hot-query layer for recent data.

Why store ip_hash not ip_address? Raw IP addresses are PII under GDPR and CCPA. Storing a one-way hash of the IP (salted and rotated) lets you perform deduplication (unique visitors) without storing the personal data itself. Mention this proactively. It signals awareness of compliance requirements.

The dominant analytics query is: "give me all clicks for URL X between time A and time B." A composite index (url_id, clicked_at) handles this perfectly. The leftmost column (url_id) filters down to a specific URL's rows, and the rightmost column (clicked_at) supports efficient range scans within that set.

Why not (clicked_at, url_id)? Reversing the order would make the index useful for "give me all clicks across all URLs in a time range", a much less common query. Always index in the order that matches your most frequent access pattern: filter first, range second.

At large scale, this index itself becomes enormous. When the clicks table moves to Cassandra, the partition key becomes url_id and the clustering column becomes clicked_at DESC, achieving the same access pattern but distributed across nodes automatically.

Redis handles most traffic well, but a single viral URL can saturate the network connection to the specific Redis shard holding that key, even if Redis itself is healthy. The L1 local cache on each API server handles this case: requests served from L1 never touch the network at all. With a 1-second TTL, each API server effectively rate-limits its own outbound Redis traffic to at most one request per key per second, regardless of incoming load.

The two layers also fail independently. If Redis goes down, L1 continues serving hot entries for up to 1 second, during which a circuit breaker can activate and route misses directly to the DB rather than hammering a failing Redis. Without L1, a Redis failure immediately becomes a DB flood.

LRU eviction handles memory pressure: when the cache fills up, the least recently used entries are dropped to make room. TTL handles correctness: entries that haven't been evicted are still expired after 24 hours, preventing perpetually stale data from accumulating for obscure URLs that were cached long ago and never requested since.

They solve different problems and you need both. LRU alone means a long-tail URL cached six months ago stays forever, even if the underlying long URL was changed. TTL alone means a popular URL that's accessed every second still gets evicted after 24 hours and causes an unnecessary DB read.

For URLs with an expiry date set by the user, override the TTL: set it to min(24h, expires_at − now). This ensures the cache entry expires no later than the URL itself, without needing a separate invalidation sweep.

URL updates and deletions are uncommon; users rarely change a short URL once created. But when they do happen, the cache must be invalidated immediately or you risk serving a stale (or deleted) URL for up to the TTL duration. The correct approach is synchronous invalidation: on any write to the urls table, issue a DEL short_code to Redis as part of the same operation (not as a background job). If the DEL fails, the write should still succeed (the TTL will eventually clean it up), but log the failure and alert.

This is a case where cache invalidation is simpler than it looks. Because short_code is the exact cache key, there's no partial invalidation, no cache stampede, and no need for versioning. Delete the key; the next request repopulates it from the DB.

As established in capacity estimation, the urls table at moderate scale (~900 GB) fits on a single node and doesn't need sharding for capacity. You shard when read throughput exceeds what a single primary + replica set can serve, or when you need geographic distribution for latency.

The shard key is short_code, using consistent hashing. The reasoning is direct: every redirect is a point lookup by short_code, so sharding on it ensures each request goes to exactly one shard, with no cross-shard queries and no scatter-gather aggregation. The alternative (sharding on user_id) would be wrong: it optimises for the management UI (low traffic) at the expense of the redirect path (the dominant operation).

A global auto-incrementing counter is the simplest ID generator but a single point of contention at scale. Two patterns eliminate that contention:

Zookeeper range allocation: each API server pre-fetches a range of IDs (e.g. 1,000–2,000) from Zookeeper at startup. It burns through them locally with no network calls. When exhausted, it fetches the next range. Zookeeper sees only one request per thousand URLs created, effectively removing it from the hot path. The risk: if a server crashes mid-range, those IDs are lost. Given the key space is 3.5 trillion, a few thousand lost IDs is irrelevant.

Snowflake IDs: a 64-bit value composed of a timestamp (41 bits), datacenter ID (5 bits), machine ID (5 bits), and per-machine sequence (12 bits). Entirely self-contained, requiring no coordination or single point of contention. The tradeoff is clock skew: if a machine's clock jumps backward, ID monotonicity breaks. Handle with a clock guard that refuses to generate IDs if the system clock regresses.

The pipeline is Kafka → Flink/Spark → ClickHouse → S3, and each stage exists for a specific reason. Kafka is the durable buffer: it decouples the API servers (producers) from the analytics processors (consumers), absorbs traffic spikes without backpressure, and retains events for 7 days allowing replay if a consumer has a bug. Flink or Spark Streaming aggregates raw click events into per-URL, per-hour, per-country counts, which is the shape your analytics dashboard actually queries. ClickHouse stores the aggregated metrics and answers dashboard queries in milliseconds due to its columnar storage format. S3 Parquet holds the raw events, partitioned by date, as the long-term archive for ad hoc queries and backfill.

Why not write directly from Kafka to ClickHouse? You could, but the aggregation step in Flink is important: it reduces 100M raw click rows/day to a far smaller set of pre-computed aggregates, keeping ClickHouse fast and cheap. Without it, every dashboard query would scan billions of rows.

GeoDNS routes users to the nearest regional cluster, reducing redirect latency from ~100 ms (cross-continent) to ~5–10 ms (local region). Each region gets its own Redis cache and a read replica of the DB. URL creation writes always go to the global primary. You never do active-active writes to avoid split-brain.

The failure mode to reason through: a URL is created in region A, and a user in region B clicks it 200 ms later, before the async replication has propagated the new row to region B's replica. The correct behaviour is not a 404 but a fallback: on a cache miss that also misses the local replica, the read path falls through to the global primary. This adds latency for that one request but guarantees correctness. P99 replication lag between regions is typically under 100 ms, so this fallback is rare in practice.