Social Post Search System Design Interview Guide

Search over user-generated posts looks, at first, like search over web pages with a smaller corpus. Tokenise the text, build an inverted index mapping each term to the posts containing it, shard it, rank the matches. That machinery is real and you will need it. But it is not what the question is about, and a candidate who spends the whole hour on it has answered a different question than the one asked.

The difference is that a web page is visible to everyone. A social post is not. A post may be public, or visible to the author's friends, or to friends except a named list, or to members of one group, or to nobody but the author. Two people running the identical query get different results, and the difference is not a matter of personalisation or ranking taste — it is a correctness boundary. Returning a post to someone entitled to see it is a feature. Returning it to someone who is not is an incident.

That single fact propagates through the entire design. It decides where the visibility check runs, and therefore how many candidates retrieval must produce to fill a page. It decides what can be cached, because a filtered result set belongs to one viewer and nobody else. It decides what "the index is stale" costs you, because everywhere else in this series stale data means a slightly wrong answer, and here it can mean a leak. The interviewer is watching for whether you notice that, and how early.

The second thing under test is whether you can reason about an index as a data structure rather than as a product you install. Some interviewers make this explicit by ruling out off-the-shelf search engines, precisely to see whether "we'd use Elasticsearch" is a design or a way of skipping one. The underlying question is the same either way: what is actually stored, what does a query do to it, and what does that cost at the corpus size you just estimated.

What the question becomes at each level

Where this post sits in the series

Several posts here border this one, and the boundaries are worth stating so you know which to read for what.

• Search autocomplete serves a precomputed top-k per prefix, built offline from query logs. Nothing is ranked at request time and every user sees the same suggestions. This post ranks at query time over a live corpus, and no two viewers necessarily see the same thing.
• Proximity search also runs a query engine over an index, but its corpus of businesses is uniformly visible to everyone, so its hard problem is spatial indexing. Strip the per-viewer filter out of this post and you are largely reading that one.
• Web crawler builds a corpus by discovering and fetching content the platform does not own. Here the content arrives on our own write path the instant it is created, so freshness is an indexing-lag problem rather than a re-crawl-scheduling problem.
• Twitter/X feed answers "what should this user see next" by fanning out along the follow graph. Search answers "find posts matching this query" across the whole corpus, with no follow edge required and no per-user timeline to fan into.
• Top-K leaderboard shares the phrase "top k" and almost nothing else: there, one numeric score dimension over a small key space; here, a multi-signal relevance score over a sharded index, computed per query.

Settle this early, because it changes the shape of the answer: we are building search over posts, not a full social-graph search engine. A query is text. Results are posts. Searching for people, pages, groups and events is a related problem with a different index and different ranking, and folding it in would double the surface without deepening anything.

Functional requirements

• Search posts by free-text query, with multiple terms treated conjunctively by default.
• Return only posts the requesting user is permitted to see, under the post's audience setting and the relationship between the two users.
• Rank by relevance, with recency as a strong signal and engagement as a weaker one; offer a "most recent" ordering as an explicit alternative.
• Paginate deterministically — a user scrolling to page three does not see page one's results again, and does not silently skip results.
• Reflect edits and deletions: a deleted post stops being findable, and an edited post is findable by its new text and not its old.

Explicitly out of scope, so the hour goes somewhere useful: media understanding (searching the contents of images and video), cross-language retrieval, data residency, and the ranking model itself. Residency is a genuine constraint at this scale — it makes a home region a partitioning key alongside time, and narrows which shards a query may legally touch — but it multiplies every section below without changing any of the decisions this post exists to make. We will design where ranking happens and how it is evaluated and rolled out, but not the feature set of the model — that is a machine learning system design question wearing this question's clothes.

Non-functional requirements

The last row deserves more weight than it usually gets. The index is not the system of record — the posts live in a primary datastore, and the index is a derived read structure built from them. That is what makes aggressive in-memory designs, lossy compression and shard-level partial results acceptable here in ways they would not be for the store itself.

The usual estimate for a storage-shaped system multiplies writes per day by row size by retention and calls the result the capacity problem. Doing that here produces a number that is large, correct, and beside the point. Index storage is a budgeting exercise. What constrains the design is how much work one query does, and that is governed by two quantities the storage math never mentions: how many shards the query touches, and how many candidates retrieval must produce before enough of them survive the visibility check to fill a page.

Start with the corpus, because it sets the shard count. Facebook's engineering team reported in 2013 that its posts index held more than one trillion posts, growing by about a billion a day. Those figures are old — treat them as the right order of magnitude for a platform of that size rather than as today's numbers — but they anchor the estimate well, and the defaults below reproduce them: a billion posts a day retained for three years is roughly 1.1 trillion posts.

Post text is small, so raw volume is modest by the standards of this series. The index is a fraction of it again: Lucene's own feature list puts index size at “roughly 20-30% the size of text indexed”, and the estimator takes the conservative end of that range. That 30% is the one constant here taken on external authority, and it comes with a caveat — it is a rule of thumb from the library's documentation rather than a measurement of any particular corpus, and the real ratio moves with your analyzer, your stored fields and your doc-values, so treat it as a planning figure, not a guarantee. §7b checks it against a published measurement on an actual corpus of short posts, decomposes what the bytes are, and reconciles the result with the much larger figure Facebook published for the same index. Everything else in the estimator is an input you can move, because the honest position is that platforms do not publish their average post length, their term counts after tokenisation, or their ratio of searches to posts.

The last two cards are the ones to sit with. A page of ten results does not cost ten candidates. It costs ten divided by the share of matches this viewer is allowed to see — and because the coordinator cannot know in advance which shard holds the survivors, every shard has to return that depth. The rows the coordinator merges is therefore shard count multiplied by retrieval depth, and both factors grow for reasons that have nothing to do with each other.

Drag the pass rate down to 2%, which is what a query matching mostly friends-only posts from strangers looks like, and the merged row count goes up by an order of magnitude while the page the user sees stays exactly ten results long. That is the whole problem in one number.

Two paths cross in this system and they have almost nothing in common. The write path is a stream: posts are committed to a primary store, and something downstream turns them into postings in an index. The read path is a fan-out: a query goes to every shard, comes back with candidates, and those candidates have to survive a permission check before anyone sees them. The only thing the two paths share is the index itself, and they contend on it — which is most of what makes §9b and §10 interesting.

The components

Search coordinator. Receives the query, broadcasts it to every shard, merges what comes back, applies the visibility filter to the merged candidates, ranks the survivors and hydrates the page. It is the only place with a global view, which makes it the only place that can rank across shards correctly — and the place where the tail latency of 1,314 parallel requests becomes one user-visible number.

Index shards. Each holds a disjoint subset of posts and the complete inverted index over that subset, so it can score its own documents without talking to anyone. The non-obvious constraint is what a shard does not know: it has no idea who is searching, and no idea how many of its matches that person may see. It returns its best candidates blind.

Visibility service. Answers one question — may this viewer see this post — against live permission state. It is a dependency of the read path, not a copy of anything inside the index, and §5 is entirely about why it is drawn this way. Its latency lands directly in the query budget, and it cannot be cached per viewer without reintroducing the staleness it exists to prevent.

Post store. The system of record for posts and their audience settings. Everything in the index is derived from it, which is what lets us treat shard loss as a rebuild rather than data loss. It also serves the final hydration: the index returns post IDs and scores, and the bodies come from here.

Change stream and indexer. Committed writes in the post store are published as a change stream, and the indexer turns each change into postings and applies them to the shard that owns the document. Facebook's posts-search team described exactly this shape in 2013, subscribing to MySQL changes through a system called Wormhole so that creates, edits, deletes and connection changes all schedule an index update. Doing this asynchronously is what decouples "the post is saved" from "the post is findable" — the gap between them is the freshness NFR.

Architectural rationale

How real systems differ here

These are not stylistic differences. Each row is a place where teams building the same kind of system made opposite choices, and each choice is traceable to something about their corpus or their permission model.

Everything so far has been the part of this system that resembles other search engines. This section is the part that does not, and it is the decision an interviewer is most likely to push on, because every instinct trained by the rest of the design points at the wrong answer.

That instinct says: push work into the index. It is why we score at the leaf instead of shipping candidates, why we denormalise, why we precompute. Applied here it says to bake visibility into the postings — index each post with the set of people who may see it, and let the query match on viewer as well as term. Retrieval then returns only permitted results, over-fetch disappears, and the visibility service leaves the read path entirely.

It is the wrong answer, and understanding why is the whole section.

Start with the naive version, because it fails in an instructive way

The simplest design does no filtering in the index at all. Ask each shard for its top ten, merge to a global top ten, check those ten against the visibility service, return what survives.

The user asks for ten results and gets four. Not because only four posts match — thousands do — but because six of the ten highest-scoring ones happened to be invisible to this viewer. Page two will be similarly holed, and the holes are in different places for different viewers.

The fix is to ask for more than you need, and the amount more is the pass rate. If a fraction p of matching posts are visible to this viewer, filling a page of K needs roughly K/p candidates. That expression is the over-fetch factor, and §3 makes it a slider because it is the number that drives everything else: at a 20% pass rate a page of ten costs fifty candidates per shard, and the coordinator merges shard count times that.

So why not put the check in the index?

Because the index cannot be fresh enough, and here staleness is not a quality problem.

Consider what an index-time design has to guarantee. When someone removes a friend, that person must immediately stop seeing the first user's friends-only posts. Every one of them, in every shard, at once. The index would have to apply a permission change across the entire corpus with the urgency of a database transaction — and an index built to absorb hundreds of thousands of postings per second in batches, with a refresh interval and a merge policy, is precisely the wrong machine for that.

The team that built Facebook's graph search reached this conclusion and documented it plainly. Their system, they wrote, "does not have privacy information incorporated into its index. Instead, our approach is to give callers all the relevant data concerning how a particular result was generated … so that the caller — typically our PHP frontend — can make a proper privacy check on the result." The justification names the exact scenario above:

"If a user 'un-friends' another user, the first user's friends-only content immediately and irrevocably must become invisible to the second user. Even a 30-second lag-time is unacceptable. … In CAP-speak, we chose availability and partition tolerance. Unicorn is not an authoritative database system."

Read that last sentence as a design principle rather than a disclaimer. The index is allowed to be stale, lossy, partially available and eventually consistent — that is what makes it fast and cheap. The moment it becomes the authority on who may see what, it inherits requirements it was never built to meet. Keeping the check outside is what lets the index stay the kind of system it is good at being.

There is a second argument, quieter but just as strong: the audience rules already exist somewhere in the product. Reimplementing them in the indexer produces two implementations that must agree forever, across every future feature that touches visibility. When they diverge — and they will — the symptom is a leak, not a failed test.

The three real answers, and what selects between them

Having argued that permissions do not belong in the index, the complication is that real systems do not agree — and they are not wrong. They have different permission models, and the mechanism follows the shape of the rule.

Our choice for this system is the third row, because the requirements in §2 describe the third row: audience settings that depend on the relationship between two users, changeable at any moment, with correctness that is binary rather than statistical. The first row does not fit because visibility here is not a property of the author alone. The second row is closer than it looks — and worth saying out loud in an interview — but a viewer's accessible set on a social platform is their friend graph, which is large, changes continuously, and would have to be compiled into every query.

Note what the third row is not. It is not "fetch everything and filter in the application", which is the strawman version. The index still prunes hard: it returns ranked candidates, and the check runs only on those. What the index does not do is claim to know the answer.

Capping the damage: truncation

Over-fetch with no ceiling is a denial-of-service waiting for the right query. The mechanism that bounds it is a truncation limit — a maximum number of candidates retrieval will consider, after which it stops and returns what it has.

The same paper measured this directly: as the truncation limit rises, latency and CPU both climb, and latency crosses 100 ms at a limit around 5,000. Their conclusion reads as an admission rather than a solution — barring architectural changes, they write, there will always be queries whose candidate sets need truncating to keep latency reasonable.

Truncation trades recall for a latency bound. A query whose visible results sit below the cut returns fewer results than exist, and the user cannot tell the difference between "nothing else matches" and "we stopped looking". That is a real defect, not a rounding error, and the only honest mitigation is to make the ranking good enough that visible results tend to sort high — which is why ranking quality and privacy filtering are coupled problems rather than independent ones.

def search(query, viewer, k=10, max_candidates=5000):
    results, scanned, cursor = [], 0, None

    while len(results) < k and scanned < max_candidates:
        # one fan-out round: every shard returns its next slice
        batch = index.retrieve(query, after=cursor, limit=k * OVERFETCH)
        if not batch:
            break                      # corpus exhausted, not truncated

        scanned += len(batch)
        cursor = batch[-1].sort_key

        # one call for the whole batch, not one per candidate
        visible = visibility.filter_visible(viewer, [c.post_id for c in batch])
        results.extend(c for c in batch if c.post_id in visible)

    truncated = len(results) < k and scanned >= max_candidates
    return results[:k], truncated

Search APIs are usually boring, and this one would be too if results were stable. They are not. Between page one and page two the corpus grows by several million posts, and — the part unique to this system — a post that was visible when page one was served may not be visible when page two is requested. The contract has to say what happens then, because the alternative is that each client invents its own answer.

The read surface is one endpoint:

GET /v1/search?q=...&limit=10&cursor=...&author=...&since=...&until=...

200 OK
{
  "results": [ { "post_id": "...", "author_id": "...", "created_at": "...",
                 "snippet": "...", "score": 8.41 } ],
  "cursor":    "eyJzIjo4LjQxLCJpZCI6..." | null,
  "truncated": false
}

Three decisions in that shape are worth defending.

The cursor is opaque and encodes a sort position, not an offset. Offset pagination re-runs the query and skips N rows, which means the cost of page ten is ten pages of work, and any insertion above the cut shifts every subsequent row. Elasticsearch caps this behaviour outright — index.max_result_window defaults to 10,000 — and its documented alternative, search_after, carries the sort values of the last row seen so the next page resumes from that position at constant cost. The same docs are explicit that the sort must include a tiebreaker, or "results could miss or duplicate hits": score alone is not unique, so the cursor carries (score, post_id) and the comparison is lexicographic on the pair.

There is no total count. Producing one means evaluating every match and visibility-checking every match, which is exactly the unbounded work §5 spent a section avoiding — and the result would still be wrong the moment it was computed. §10b gives the second, better reason to omit it: a per-viewer result count is a measurement of content the viewer cannot see, which makes it a privacy leak in the shape of a UI affordance.

truncated is part of the response, not a log line. When retrieval hits the candidate ceiling before filling the page, the client is told. A UI that knows the difference between "that's everything" and "we stopped looking" can offer to search deeper or narrow the query; a UI that cannot tell shows the user an empty state that is a lie.

The dominant path is the read. Tracing one query through the components in §4 shows where the time actually goes, and — more usefully — shows that the answer is not what the diagram suggests.

1. Parse and plan. The coordinator tokenises the query the same way the indexer tokenised documents — an asymmetry here is a silent recall bug, not an error — applies the filters from §5b to decide which shards and which time-ranged segments can be skipped, and sets a retrieval depth from the last observed pass rate for this viewer class.

2. Fan out and score. Every surviving shard walks its postings lists, intersects them, scores the matches and returns its own top slice. Scoring at the leaf rather than shipping candidates upward is what keeps the merge affordable, and it is the same reasoning §4 used for partitioning by post rather than by term.

3. Merge. The coordinator takes shard-count × depth rows — 65.7k at the estimator's defaults — and reduces them to a ranked candidate list. This is a heap merge over already-sorted runs, which is cheap per row but not free at that volume.

4. Filter. The top candidates go to the visibility service in one batched call. Survivors fill the page; if the page is short and the candidate budget is not exhausted, the coordinator takes the next slice from the merged list and checks again.

5. Hydrate. The ten survivors — and only the ten — are fetched from the post store for text and snippet generation. Everything before this step moved identifiers.

That budget contains the section's real point. The single largest line is not the visibility check that §5 spent so long on; it is the gap between the typical shard and the slowest one. Dean and Barroso quantified exactly this in The Tail at Scale: if one request in a hundred is slow, fanning out to a hundred servers means 63% of user requests are slow. Their measured example is sharper still — a service whose individual leaves have a 10 ms p99 shows 70 ms latency when 95% of leaves have responded, and 140 ms when all of them have. The last 5% of shards double the query time.

At 1,314 shards, "the slowest of N" is not a tail effect to be tuned away. It is the dominant term, and it is why §9 treats shard count as a quantity to be actively reduced rather than a number that falls out of the storage math — and why §10's degradation ladder starts with returning results from the shards that answered.

There are four things worth modelling here — the post, the inverted index over posts, the audience rule attached to a post, and the relationship graph the rule is evaluated against — and the temptation is to put as many of them as possible in one place. The access patterns say otherwise.

Two observations force the shape. The first is that every row in that table is keyed by something different: the index by term, hydration by post id, the visibility check by a pair of users. No single store is good at all three, and combining any two of them means one access pattern runs as a scan. The second is the last row. An audience change must take effect immediately and must not require touching the index — which is not a storage optimisation, it is §5's decision expressed as a schema constraint. The moment the index holds a copy of the audience rule, changing the rule means rewriting the index.

The inverted index, and why it is built out of segments

The post store answers one question well: what does post 7 say? Search asks the opposite question — which posts say "coffee"? — and answering that from the post store means reading every post. The inverted index is that table turned around: instead of post → words, it stores word → posts. That inversion is the whole trick, and everything else in this section is a consequence of it.

A postings list holds three things per matching post, and §7b prices each of them. The document id, stored as a gap from the previous id rather than an absolute — which is why the list has to be walked in order rather than indexed into. The term frequency, for scoring. And the positions the term occupies in the text, which is what makes a phrase query possible and what a proximity match reads. Document ids are the index; the other two are features you can decline to pay for.

So far this describes a structure you build once. The harder problem is keeping it current while a billion posts a day arrive, and the answer every production engine converges on is counter-intuitive: never modify the index at all. A term's postings list is a tightly packed run of bytes, so inserting one document in the middle of it means rewriting the run — and a single post touches twenty such lists. Instead, the index is cut into segments. A segment is a complete, self-contained miniature index over a slice of the documents: its own dictionary, its own postings, its own sidecar. Once written, it is never changed.

Immutability buys three things at once. Writes stop being edits and become appends to the one small mutable segment, which is flushed and frozen on the freshness interval §9b sets. Readers never need a lock, because a segment being merged is not the segment they are reading. And caching becomes trivial, since a file that cannot change cannot go stale.

It is paid for in three matching ways, and each one surfaces later in this article. A query has to search every segment and merge the results, so segment count is a latency term — which is why the system merges continuously, and why §9b treats merge throughput as a budget rather than a background detail. A delete cannot remove anything, so it flips a bit in the segment's live-documents bitmap and the document is skipped as the postings are walked; the space comes back only when that segment is merged. And an edit is a delete plus an insert, which is why the edit-version check below matters and why edited text lingers until a merge rewrites it.

Concretely, a segment on disk is a handful of files, each answering a different lookup:

segment/
  terms.dict     term        -> offset into postings
  postings.bin   term        -> [doc_id delta, term_freq, positions...]
  docvalues.bin  doc_id      -> created_at, author_id, audience_kind
  livedocs.bits  doc_id      -> 1 live | 0 deleted
  ids.map        doc_id      -> post_id          # segment-local, ephemeral

The fields in docvalues are exactly the ones §5b's filters need to prune with and the merge needs to sort by — nothing else. Post text is not among them: it lives in the post store, and hydration fetches it for the ten results that survive.

Note what audience_kind is and is not. It records the category of the rule — public, friends, custom list, group-only — not who may see the post. That is enough for a fast path: a query filtered to public posts can skip the visibility service entirely, because the pass rate for public content is 1 and there is nothing to check. It is not enough to decide anything about a non-public post, which is the point.

The post store and the graph

The post store is a partitioned key-value store keyed by post_id, holding author, timestamp, text and the full audience descriptor. It is the system of record for post content; the index is derived from it, which is what makes §2's "rebuildable, not durable" requirement true. The relationship graph is not part of this system at all — the visibility service owns it, and this design deliberately holds no replica of it, because a replica is a staleness window with extra steps.

The correctness problem nobody mentions: deletes and edits

A post's lifecycle is where a search index quietly goes wrong, and the ordering of two writes decides whether the failure is visible or dangerous.

Deleting a post means removing it from the post store and clearing its bit in the live-documents bitmap of whichever segment holds it. Those are separate writes to separate systems, so one lands first. Clear the index bit first and a crash leaves a post that exists and is unfindable — bad, and self-healing on the next reindex. Delete from the post store first and a crash leaves an index entry pointing at nothing, so hydration fails and the coordinator drops the result. Dropping it is the correct behaviour: a result that cannot be hydrated is not returned. So the post store is deleted first, and hydration failure is treated as a filter rather than an error.

Editing is subtler, because an edit produces a window in which the index describes text that no longer exists. A post edited to remove a word still matches that word until its segment is rewritten, and the snippet generated from the current text will not contain the term the user searched for. That is a quality bug, not a privacy one — and it is worth saying which it is, because the reflex is to treat all index staleness as equally serious.

There is a third case that neither ordering covers, because it arrives through the retry path rather than the crash path. The change stream delivers at least once, so an indexer that dies mid-batch sees some events again when it restarts. Applying a posting therefore has to be idempotent rather than additive: postings carry the post's edit version, a redelivered event overwrites the posting it already wrote instead of appending a second one, and an event carrying a version older than the segment already holds is dropped rather than applied. Without that version a redelivered edit can reinstate text the author has already replaced — the same defect as the delete-ordering bug, reached by a different route.

The privacy-relevant edit is a change to the audience rule, and this design has already solved it. Because the rule lives in the post store and is evaluated at query time, tightening a post's audience takes effect on the next query with no index write at all. That is the dividend from §5 that shows up nowhere in the query path and everywhere in the failure analysis.

§3 sized the index at 65.7 TB and moved on, because what binds this system is query cost rather than capacity. It is worth returning to now that §7 has named the files. Index size sets shard count, and §6 established shard count as the dominant latency term, so this is a latency input wearing a storage costume — and the one published figure for a real system of this shape is more than ten times larger.

Where the bytes actually go

A posting is one (term, document) pair. At the estimator's defaults — 200 bytes of text, 20 indexed terms — 65.7 TB across 1.1 trillion posts is 60 bytes per post, or 24 bits per posting. That rests on a documentation rule of thumb, so it is worth breaking apart and checking against something measured on real short text.

Three things fall out of that split. Document ids are nearly two thirds of the index and are not a lever — they are the matching. Positions are the only large component tied to a single feature, which makes them the one real choice on the table. And term frequencies are nearly free here, because a 200-byte post rarely repeats a term, so the counts compress to almost nothing.

The estimator's implied 24 bits per posting lands essentially on the 23 measured for Lucene on that corpus, which is reassurance rather than luck: the 20–30% rule is Lucene's documentation describing Lucene's own output. Newer codecs beat that figure, so the estimate errs high — the direction an estimate should err.

Why the published figure is ten times bigger

§12 cites Facebook's 2013 report of more than a trillion posts in more than 700 TB — roughly 640 bytes per post, against the estimator's 60 for the same corpus size. Same platform, same scale, an order of magnitude apart. What differs is not the compression. It is what is being counted.

65.7 TB is the inverted index over post text: the structure a query walks. The 700 TB is an entire serving footprint, and Meta's own write-up says what fills it — dozens of distinct kinds of data sorted and indexed on, well over a hundred ranking features, and the document data held separately from the inverted index. Per-document ranking data at that richness dwarfs the text postings, and none of it is the text index.

This design made the opposite choice explicitly, in §7: docvalues holds created_at, author_id and audience_kind and nothing else, because those are what §5b's filters prune with and what the merge sorts by. Text and the full audience descriptor sit in the post store, reached by point read for the ten survivors. The two numbers measure different objects, and §3's shard count stands.

What medium holds it

§6 gives each shard about 20 ms to walk its postings, and a multi-term query touches on the order of twenty posting lists. That budget, divided that way, picks the medium before cost is even considered.

Two rows are disqualified on latency before cost is considered at all, and the two that survive are separated by roughly thirty times in price. That is the whole decision, and both Facebook and Twitter documented making it on indexes of exactly this shape — bulk postings onto flash, the hottest structures held back in RAM.

What drove Facebook's move is the part worth carrying, because it is not the obvious thing: not the price of memory, but the coordination overhead of spreading an index across enough machines to hold it — the same quantity §6 prices as fan-out. Twitter was blunter about what it cost them, taking a major hit to per-machine query capacity to move its archive off RAM, and taking it anyway.

That is what §9's age tiering buys. The recent tier is touched by every query and earns page cache over local flash; the archive is touched only when a date filter overlaps it, which makes storage two orders of magnitude cheaper and three orders slower an acceptable trade. Seconds-scale p99 for a sixteenth of the cost is a bad number for interactive search and a fine one for "find the thing I posted in 2019".

Tiering pays a second time, and this is where §2's classification of the index as derived state cashes out. Replicas of a search index buy read throughput and recovery time, not durability — the post store is the system of record, and §9b describes rebuilding from it. So replica count is a performance decision that can differ per tier: the recent tier carries replicas because it absorbs every query, while an object-backed archive can carry none, since a lost node's cache refills from a store that is already redundant. Elastic puts that saving at roughly half the disk of an equivalent replicated tier. Sizing every tier at 3× because "that is what replication means" is the common way to get this wrong.

Working the tradeoff in an interview

When an interviewer pushes on size, the weak answer is a smaller estimate. The strong one is a named lever with a number on it, and an order to reach for them in. These are the levers that exist.

The ordering carries as much signal as the levers. Take the codec win first because it costs nothing. Then decide about positions, which is the only structural choice in the list and the one that turns on a product question rather than an engineering one — is quoted search a headline feature or a long-tail convenience? Tier by age next, because recency skew is real here and the archive is where the bytes are. Reach for shard count last, and say why: §6 already showed that fan-out, not bytes, is what the latency budget cannot absorb.

The read path in §6 spends 177 ms doing work, most of it repeated across queries, and a cache is the obvious response. It is also the place where this design is most likely to be quietly broken by someone optimising in good faith, because the most valuable-looking thing to cache is the one thing that must not be.

The test to apply to each candidate is not "how often does this change?" but "what is the consequence of serving it stale?" — and in this system those two questions have different answers for different data.

The last row is §5's argument arriving in a different costume. A cache of visibility decisions with a sixty-second TTL is functionally an index with a sixty-second staleness window, and the reason to reject it is the one the Unicorn authors gave for rejecting the index: after an unfriend, "even a 30-second lag-time is unacceptable". A cache does not become acceptable by being called a cache.

The fourth row is the interesting one, because it looks like the fifth and is not. Caching a post's audience descriptor memoises a property of the post — "this post is visible to the author's friends" — which changes only when the author edits it, and can therefore be invalidated by the write that changes it. Caching the decision memoises a property of the pair, which changes when either side's relationships change, with no write to this system to hang an invalidation on. Same latency saving, completely different correctness story.

That distinction is what makes the visibility check affordable without weakening it. The expensive part is the reachability evaluation, and it is the authoritative service's job to make that fast — a friend-set membership test against data it owns and can invalidate. This system caches the inputs it is allowed to cache and asks for the answer every time.

§6 identified the dominant latency term, and it was not the part of the system this article has spent the most words on. It was the fan-out: waiting for the slowest of roughly 1,314 shards. Scaling this system means attacking that number, and then dealing with the two problems that attacking it creates.

Reducing the number of shards a query touches

There are only two ways to touch fewer shards: make each shard hold more, or route the query to a subset.

Bigger shards have a ceiling that is not about storage. Elastic's guidance puts the working range at 10–50 GB per shard, and the constraints behind it are recovery time, merge cost and heap pressure rather than capacity — a shard is the unit of replication and rebalancing, so a very large one makes every failure slower to recover from. Pushing from 50 GB to 200 GB would cut the fan-out to about 330, which is a real improvement, at the price of quadrupling how long it takes to rebuild a lost shard.

Routing is the better lever, and it works here because of something true of post search specifically: queries are overwhelmingly recency-biased. People search for the thing they saw last week far more often than for something from 2019. That makes time a partitioning dimension the query knows in advance, which is exactly the property §3's callout said routing requires.

So the index is tiered by age. A recent tier — the last few weeks — is touched by every query and is therefore held on the media §7b showed can serve a 20 ms shard: page cache over local flash. An archive tier holds everything older, partitioned by time range, and a query touches only the ranges its since/until filters overlap. An unfiltered query still has to consider the archive, but it can consult the recent tier first and skip the archive entirely once it has enough high-scoring recent results.

Routing and storage cost are the same argument reaching the same conclusion from two directions. Routing makes the archive rarely touched; being rarely touched is what makes it affordable to put on storage two orders of magnitude cheaper and three orders of magnitude slower. Neither half works alone — a cheap archive that every query reads is just a slow index.

Twitter's Earlybird shows the shape at the segment level: only the newest segment per server is actively written, and the rest are optimised into a compact read-only form — which is what makes holding many of them per server affordable.

The problem tiering creates: scores stop being comparable

This is the part most write-ups skip, and it is a genuine correctness issue rather than a tuning detail.

Relevance scoring weights a term by how rare it is — a match on "photosynthesis" means more than a match on "the". Rarity is measured as inverse document frequency, and document frequency is counted per shard, because counting it globally would require a round trip before the query could start. Elasticsearch's default search type, query_then_fetch, does exactly this: each shard scores using its own local statistics, and the coordinator merges scores that were computed against different denominators.

With documents distributed randomly across many shards this is close to harmless, because every shard's local frequency approximates the global one. Two things break that assumption, and this design has both. Rare terms are the first: a term appearing in three documents platform-wide may appear in one shard and not its neighbour, so the same document scores differently depending on where it landed. Non-random partitioning is the second, and it is the one tiering introduces — once shards are partitioned by time, terms that were common last year and rare this month have wildly different local frequencies, and a document's score depends on which tier it is in.

The available fix is dfs_query_then_fetch, which adds a preliminary round trip to gather global term statistics before scoring. Elastic's documentation describes it as slower but more accurate, and the cost is precisely the thing §6 is trying to protect: an extra fan-out to 1,314 shards before the real one begins.

The proportionate answer is not to enable global statistics for every query. It is to keep the distribution random within a tier, so local statistics stay representative for the common case, and accept score drift across tiers as the price of the routing that makes the system fast. Where it matters — a small number of head queries where ranking quality is worth the latency — the global-statistics path can be turned on selectively. Turning it on everywhere buys accuracy the users cannot perceive at a latency cost they can.

Staying fresh: one writable segment, many frozen ones

§2 asks for a new post to be findable in about ten seconds, and §3 says 231.5k postings per second are arriving. Immutable segments and a ten-second freshness target are in obvious tension — an immutable structure cannot absorb a write.

The resolution is that exactly one segment per shard is mutable. New postings append to an in-memory segment that is optimised for write throughput rather than compactness; queries search it alongside the frozen ones and merge the results. Periodically it is sealed, optimised and becomes read-only, and a fresh one takes over. Elasticsearch exposes this as the refresh interval, defaulting to one second — which is the knob that decides how quickly a write becomes visible, and the single most useful thing to name when an interviewer asks how near-real-time search works.

Earlybird is the worked example: its measured end-to-end ingest latency was around ten seconds, with queries around fifty milliseconds — which is where §2's freshness target comes from.

Search is allowed to fail. §2 sets availability at 99.9% and calls a failure degraded rather than lost, which gives this system a luxury the write path does not have: it can return less rather than returning wrong. The design question is what "less" means at each step down, and the ordering matters, because one of these failures has a consequence the others do not.

The last two rows are where the levels separate. Every row above them is a capacity or availability failure, and the response is some version of doing less work. The visibility rows are correctness failures, and they behave differently in one specific way: they do not announce themselves. A slow shard shows up in a latency graph. A visibility service that has started answering "yes" too often shows up nowhere — latency is fine, error rate is zero, the product works, and the only signal is content reaching people it should not.

That asymmetry is the argument for continuous verification rather than monitoring. Canary posts with known audiences, queried by accounts known not to be in those audiences, turn a silent correctness failure into an alert. It is a cheap mechanism and it exists because nothing else in the stack can detect the failure at all.

The abuse model for most systems in this series is borrowed from the general user-generated-content checklist — spam, scraping, rate limits. This system has one that is specific to it, and it survives every mechanism §5 put in place.

A search engine that filters results per viewer is, structurally, an oracle. The attacker cannot read the hidden posts, but they can ask questions whose answers depend on them, and the system answers honestly because it was built to. Filtering the results is not the same as filtering the information the response carries about the results.

Three channels that leak

Result counts. A total-matches number computed before filtering tells the viewer how many posts exist that they cannot see. Computed after filtering it is expensive and still differential — the count changes when a hidden post is created. §5b omits the count entirely, and this is the second reason why.

Timing. This one is harder, and it is worth admitting rather than solving badly. A query matching nothing at all terminates immediately: the postings lists are empty and there is nothing to check. A query matching two hundred posts, all invisible to this viewer, walks those postings, scores them, merges them, batches them to the visibility service, and gets zero survivors — running the entire pipeline to return the same empty page. The responses look identical and take measurably different amounts of time, so an attacker who can search a distinctive phrase can distinguish "no such post exists" from "a post exists that you may not see."

The truncated flag. §5b introduced it as an honesty mechanism, and against an attacker it is the cleanest leak of the three: truncated: true with an empty result list says directly that the system found candidates and none of them were visible. The flag that makes the product honest makes the oracle precise.

What to actually do about it

The last one has a clean fix, and it is the kind of detail an interviewer remembers: report truncation only when the page is non-empty. A response with zero results is always a plain empty result set, whatever happened internally. The user who got zero results has nothing to act on anyway — "search deeper" is not a useful offer when nothing surfaced — so the signal is worth nothing to them and everything to an attacker.

Timing has no clean fix. Padding every response to a fixed duration means every query pays the worst case, which contradicts §2's latency target outright. The proportionate mitigations are to rate-limit per account so probing costs more than it yields, to add jitter that makes single-query inference unreliable without making the median slower, and to detect the access pattern — a sequence of narrowly varying phrase queries returning zero results is not what search traffic looks like.

The other two, briefly

Expensive queries as a denial of service. §3 made the cost of a query a function of shards touched and retrieval depth, and both are attacker-influenceable: an unfiltered query over rare terms with a low pass rate is the most expensive thing this system can do, and it costs the attacker one HTTP request. Rate limiting by request count prices every query the same and therefore prices none of them correctly. The mechanism that fits is cost-based admission — estimate shards × depth before executing, charge it against a per-account budget, and reject or degrade when the budget is spent. Deep pagination is the same problem with a cursor attached, which is why §5b's cursor is signed.

Index poisoning. Stuffing a post with unrelated popular terms to appear in searches for them is the oldest attack on any index. Part of the defence is already present for unrelated reasons: BM25's term-frequency saturation (§9) means repeating a word twenty times does not score twenty times higher, so the crudest version of the attack fails against the default scoring function. The rest is ordinary — spam signals in ranking, per-author indexing rate limits, and treating a sudden vocabulary shift in one author's posts as a signal.

Deletion that is actually deletion. §7's tombstone hides a document from queries; it does not remove its text from the segment file, which keeps it until a merge rewrites the segment. For an ordinary delete that is fine. For a legal erasure request it is not — the obligation is that the content is gone, not unfindable. The mechanism is to force a merge of segments containing erasure-marked documents inside the compliance window, and to remember that the index is not the only copy: caches (§8), snapshots, and any log that captured post text are all in scope. The post store delete is the authoritative act; everything else is cleanup with a deadline.

Almost every candidate draws the same three boxes: a coordinator, a set of index shards, a document store. The levels do not separate on the diagram. They separate on when the candidate notices that a result set has to be filtered per viewer, and what they do once they have noticed — because that single observation reshapes the capacity model, the API, the caching strategy and the failure analysis.

Classic probes

Each of these is either published by the team that measured it or derivable from the estimator in §3. The dates matter — several are a decade old, and they are included as the right order of magnitude for a platform of that size rather than as current figures.

Scale and sizing

Latency and the fan-out tail

The two that decide the design