Object Storage (Amazon S3) System Design Interview Guide

The interface is almost insultingly simple. PUT some bytes under a name, GET them back, LIST names by prefix, DELETE. There are no joins, no transactions across keys, no in-place updates. Candidates who have used an object store every day of their career tend to sketch a load balancer, a metadata table and "a bunch of storage servers" in the first five minutes and then run out of things to say. The question is hard everywhere the interface hides it.

With a hundred thousand disks, a disk dies every few hours, so durability is not a property of where you wrote the bytes; it is a race between failure and repair, and you have to win it for a trillion objects while paying for well under three copies of each byte. Three copies is the answer everyone reaches for, and on paper it is enough. The interview begins when the interviewer points at the bill: at exabyte scale the difference between 3× and 1.8× storage overhead is 75,000 hard drives. Getting under 3× means erasure coding, erasure coding means repairs that read ten fragments to rebuild one, and spreading those fragments so that a whole availability zone can burn down without losing anything puts a hard floor under how cheap the code can get.

The second thing being tested is whether you notice that the metadata is its own system. Every object has a record: its key, its size, its checksum, where its bytes live. At a trillion objects that index is more than a petabyte, it must return keys in lexicographic order for LIST, and it must be strongly consistent so that a GET issued the instant after a PUT sees the new object. That is a distributed database problem sitting beside a storage problem, and the two want opposite hardware, opposite replication, and opposite partitioning. An answer that stores metadata and bytes in the same place has merged the two hardest parts of the design into one that does neither well.

This post is about the blob store itself. It is not about keeping a user's laptop in sync with the cloud: chunking a file, uploading only the changed blocks, deduplicating across users and resolving edit conflicts are covered in the Dropbox and Google Drive guide, which assumes the store this post designs. The metadata index here is a key-value store in its own right, and this post deals with how it is partitioned and kept consistent rather than how it stores bytes on one machine; the storage engine underneath, LSM trees, compaction and quorum replication, is the subject of the key-value store guide.

The functional list is short because the interface is small. What matters is the precise semantics of each operation, since every one of them is a promise that the design later has to keep under failure.

• PUT, GET, HEAD, DELETE by bucket and key. An object is an immutable blob of 0 bytes to 5 TB plus a small amount of metadata. Overwriting a key replaces the whole object atomically; readers see the old object or the new one, never a mix. GET supports byte ranges.
• LIST by prefix. Keys come back in lexicographic order, a page at a time, with an optional delimiter so that photos/2026/ can be browsed like a folder even though no folder exists.
• Multipart upload. Large objects are uploaded as independently retried parts, in parallel and in any order, and become visible as a single object only when the upload is completed.
• Versioning, per bucket and optional. When enabled, an overwrite or delete keeps the previous version retrievable instead of destroying it.
• Storage classes and lifecycle rules. Objects can move to cheaper, slower classes as they age, and can expire, on rules the bucket owner sets.
• Access control. Every request is signed; buckets carry policies; a time-limited presigned URL can grant one operation on one key to someone without credentials.

The non-functional targets are where the architecture comes from. Each one is used by at least one later decision, and the accordions below say which.

What each target forces

Most capacity estimates start from requests per second. For an object store that is the wrong first question, because the aggregate request rate turns out not to bind anything. The quantities that decide this design are stored bytes, which set how many disks exist and therefore how often one fails, and the number of objects, which sets the size of the index independently of how many bytes those objects hold.

Take one large region as the design point: 1 EB of logical data, an average object of 1 MB, so about a trillion objects. For scale, AWS has said S3 holds more than 400 trillion objects across all its regions, so a trillion in one region is a big region rather than an imaginary one. The average hides a heavily skewed distribution: most objects are small, and most bytes sit in a minority of large ones. That skew matters twice later on. It is why small objects are packed into large extents before being erasure coded (§5), and it is why the index has to be sized by object count, which small objects inflate, rather than by bytes.

So the dimensions that bind are: logical bytes and the erasure code, which together give raw capacity and the disk count; the disk failure rate, which turns the disk count into a steady stream of repairs and a repair bandwidth; and the average object size, which turns bytes into index rows. Request rate is left out of the sliders on purpose. At a peak of 1 M requests a second, 90% of them reads, the index sees about 100 requests a second per partition across the roughly 10,000 partitions sized below, and the disks see 0.9 M reads a second against about 11 M random IOPS of capacity. In aggregate there is headroom everywhere. Request rate binds only locally, on one hot prefix or one hot disk, and §9 is where that gets handled.

§1 argued that there are two systems here, and §3 put numbers on both: 1.5 PB of small, mutable, strongly consistent records, and 1.8 EB of large, immutable, cost-dominated bytes. The architecture keeps them apart and puts one thin layer between them. Everything on the request path is either stateless or replicated across all three zones; everything that moves bytes around after they are written runs in the background, off any request path.

Client / SDK. Signs every request with a key derived from the caller's secret, and splits large uploads into parts. The non-obvious constraint is that the client is also the retry policy: a throttled or failed request is retried from the SDK with backoff, so the error codes in §5b are part of the design, not an afterthought.

Load balancers. DNS spreads clients across all three zones, and a load balancer in each zone spreads connections across front ends. They know nothing about keys; any front end can serve any request, which is what lets a whole zone's front ends disappear without clients noticing more than a reconnect.

Front-end fleet. Verifies the signature, evaluates the bucket's access policy, then coordinates the operation: find or commit the object record, find or write the bytes. It streams bodies rather than buffering them, because an object can be 5 GB in a single PUT. Stateless apart from caches, which §8 is careful about.

Metadata index. One record per object version, keyed by bucket and key, stored in key order and split into roughly 10,000 ranges. Each range is a small consensus group with one replica per zone, which is what gives strong consistency and zone-loss tolerance for records. The non-obvious constraint is ordering: LIST needs keys in lexicographic order, so the index cannot hash keys to partitions, and inherits the hot-range problem that §9 deals with.

Extent manager. The control plane for bytes. Objects are packed into large append-only extents, and the extent manager records, for every extent, whether it is open or sealed, how it is encoded, and which drive holds each replica or fragment. It chooses placements and schedules sealing, coding and repair. It is on the request path only on a cache miss, since front ends cache extent locations.

Storage nodes. Servers holding a few dozen hard drives and a small SSD used as a write journal. They store extent replicas and fragments, checksum every block they write, verify on every read, and know nothing about buckets or keys. That ignorance is deliberate: a storage node can be drained, rebuilt or replaced without anyone touching the index.

Background workers. The erasure coder converts sealed extents from three replicas into RS(10,8) fragments. Repair rebuilds fragments lost with a drive, and the scrubber re-reads every fragment on a cycle to find corruption before a reader does. Garbage collection rewrites extents that are mostly deleted data. Lifecycle moves or expires objects by rule. Every one of them changes where bytes live; none of them is allowed to slow a request.

Architectural rationale

How real systems made these decisions

Every other decision in this design is about a few petabytes of index or a few thousand servers. This one is about 1.8 EB, the drives that hold them, and how many of those drives can fail before something is gone. There is no fork between competing algorithms here. There is one idea, redundancy, and a sequence of refinements, each forced by a requirement the previous version fails.

Step 1: three copies, one per zone

The obvious design writes each object to three drives, one in each zone. It works, and it deserves credit before it gets criticised. A zone loss leaves two copies, so reads continue and nothing is lost. A healthy read touches one drive. Repairing a lost copy reads exactly one surviving copy. Under the independent failure model from §3, it gives about 11.4 nines, just past the target.

Its problem is the bill: 3× raw per logical byte, against a ceiling of 2× in §2. At 1 EB that is 187,500 drives, and the margin it buys, a single spare copy once a zone is gone, is thinner than it looks.

Step 2: any k of n

An erasure code splits data into k data fragments and computes m parity fragments from them, such that any k of the n = k+m fragments are enough to rebuild the rest. Reed-Solomon codes have exactly this property. A RS(10,4) code stores fourteen fragments for every ten fragments' worth of data, 1.4× overhead, and survives the loss of any four. Three-way replication, by comparison, is the degenerate code with k = 1 and m = 2: 3× overhead, survives any two.

The codes used in storage are systematic: the ten data fragments are simply the original bytes cut into ten pieces, and only the parity fragments are computed. A healthy read of an object inside data fragment 3 goes to one drive and reads the bytes directly, with no decoding at all. Decoding happens only when a fragment is missing, and then it is expensive: a degraded read fetches the same byte range from ten surviving fragments and reconstructs the missing one, and a repair does the same for the whole fragment. Ten reads to recover one is the price of a ten-wide code, and §3's coral card showed it, for the RS(10,8) this section settles on, as 11.4 GB/s of repair traffic that never stops.

Step 3: the zone rule sets the parity count

Now apply the requirement from §2 that a zone can be lost outright. A stripe survives a zone loss only if the zone held no more fragments than the code can lose. Spread n fragments evenly across three zones, a per zone, and the rule is m ≥ a. That gives a floor: overhead n / k is at least 3a / 2a = 1.5× for any code spread over three zones, no matter how clever. RS(10,4) at 1.4× cannot be made zone-safe; a zone would hold five fragments of a code that can lose four.

At exactly 1.5× the code survives a zone loss with no margin left: the two surviving zones hold exactly k fragments. Re-protecting a destroyed zone means recreating a third of the raw bytes, 500 PB at 1.5×, by reading about 1 EB from the survivors, and placing the results without breaking the zone rule. That is days of work even with the whole fleet on it, and with several drives failing every day, some of the billion zero-margin stripes will certainly lose a fragment during those days. So the code needs margin after a zone is gone, and every fragment of margin is paid for in overhead.

Our choice for this system is RS(10,8), six fragments per zone, one per rack. It keeps two spare fragments after a zone is lost, so a stripe in a surviving zone can lose a drive, be flagged, and be repaired first while it still has one spare left. RS(14,10) buys the same margin for 0.09× less overhead, about 5,400 fewer drives at 1 EB, but costs 40% more reads per repaired fragment, about a third more repair traffic fleet-wide, puts fourteen drives behind every degraded read, and needs eight separate racks per zone for every stripe. At this scale the cheaper repair wins. Notice what did not set the parity count: the disk failure rate. §3's durability card shows that three parity fragments would already clear eleven nines against independent failures. Eight exist because a zone holds six.

Step 4: do not erasure code on the PUT path

The last refinement is about when to code. Coding every object as it arrives would make each PUT wait for eighteen drives in three zones to acknowledge, and a request that waits on eighteen drives waits on the slowest of them, which at p99 is far slower than the slowest of three. It would also shred small objects into slivers, which §4 already ruled out. So new data takes the Step 1 path for a short while: PUTs append to an open extent replicated on three drives in three zones, and when the extent reaches 1 GB (or a time limit, so slow-filling extents do not stay replicated forever) it is sealed. The erasure coder then reads the sealed extent, writes eighteen fragments under the placement rule, verifies them, flips the extent map entry, and only then frees the three replicas. The replicated window is short enough that §3 could ignore its cost.

FRAG = 100 * MB           # 1 GB extent / 10 data fragments
K, M, PER_ZONE = 10, 8, 6

def place_stripe(extent_id, zones=("A", "B", "C")):
    """Choose 18 drives: 6 per zone, never two on one rack, coolest first."""
    chosen = []
    for zone in zones:
        racks = eligible_racks(zone)                  # healthy, below fill target
        racks.sort(key=lambda r: r.recent_iops)       # heat, not just free space (see §9)
        picks = [coolest_drive(r) for r in racks[:PER_ZONE]]
        if len(picks) < PER_ZONE:
            raise PlacementError(f"{zone}: fewer than {PER_ZONE} eligible racks")
        chosen += picks
    return chosen                                     # fragment i lives on chosen[i]

def read(extent, offset, length):
    """Assumes the range sits inside one data fragment; real code splits it."""
    i, off = divmod(offset, FRAG)                     # systematic: bytes live in D(i+1) as-is
    try:
        return fetch(extent.loc[i], off, length, timeout=READ_TIMEOUT)
    except (DriveDown, Timeout):
        # degraded read: same byte range from any 10 other fragments, then decode
        others = [j for j in range(K + M) if j != i and healthy(extent.loc[j])]
        if len(others) < K:
            raise Unavailable(extent.id)
        parts = fetch_all([(extent.loc[j], off, length) for j in others[:K]])
        return rs_reconstruct(parts, missing=i)

An object store is an infrastructure primitive, so its API matters less as a set of routes than as a set of promises. Other systems are built on the exact meaning of "PUT returned 200", and on what a client is supposed to do with each error. The operations are shaped like S3's because most readers already know them; the interest is in the guarantees attached to each.

A PUT carries its own checksum, and a conditional header turns it into a create-if-absent:

PUT /media-prod/photos/2026/09/17/cat.jpg HTTP/1.1
x-amz-checksum-crc32c: yZRlqg==           // verified end to end before commit
If-None-Match: *                           // create only; 412 if the key exists
Content-Length: 1048576

<1 MB of bytes>

200 OK
ETag: "9b2cf535f27731c974343645a3985328"
x-amz-version-id: 3sL4kqtJlcpXroDTDmJ.rmSpXd3dIbrHY     // versioned buckets only

LIST is where ordering becomes visible. With a delimiter, keys that share a prefix up to the next / are rolled up into a single CommonPrefixes entry, which is how a flat key space is browsed as folders:

GET /media-prod?list-type=2&prefix=photos/2026/09/&delimiter=/ HTTP/1.1

200 OK   (abridged)
  IsTruncated: true,  NextContinuationToken: 1ueGcxLPRx1Tr...   // opaque: encodes the last key returned
  CommonPrefixes: photos/2026/09/16/, photos/2026/09/17/
  Contents:       photos/2026/09/index.json (2,048 bytes)

The guarantees, stated precisely

• Durable on 200. A PUT or a completed multipart upload does not return success until its bytes are on three drives spread across the zones (one per zone when all three are healthy) and its record is committed by a majority of the index partition's three zone replicas, so either survives the loss of any one zone. There is no "accepted, will be durable soon" state.
• Atomic visibility. The new version becomes visible in one step, at the index commit (§6). A reader sees the old object or the new one, never a partial upload.
• Strong read-after-write. Any GET, HEAD or LIST that starts after a write returns success reflects that write, whichever front end and zone serves it.
• Concurrent writers. Two PUTs to the same key are serialised by the key's index partition, and whichever commits last is the current version. A client that needs compare-and-swap sends If-Match: "<etag>" and gets 412 if someone else got there first. S3 added exactly these conditional writes in 2024.

Errors, and what the client does with each

Optional capabilities, by level

A PUT touches two independent systems, the storage nodes and the index, and both writes can fail. The order they happen in decides what a failure leaves behind, which is why the PUT is worth tracing step by step. The rule is simple and absolute in intent: bytes first, record second. A crash between the two leaves bytes that nothing references, which is waste. The opposite order would leave a record pointing at bytes that do not exist, which is data loss as far as the customer can tell.

Steps ① and ② are the front end doing its job as a gatekeeper. Step ③ is the only place bytes move: the front end streams the body to the primary replica of an open extent it holds a lease on, and the primary forwards it down a chain to replicas in the other two zones. Each replica writes to its SSD journal and fsyncs before acknowledging; the hard drive is written later from the journal. An append succeeds only if all three replicas acknowledge. If one is unreachable, the front end does not wait for it or vote around it: the extent is sealed at its current length, and the append is retried on a fresh open extent with three healthy replicas. Refusing to write to a degraded extent keeps the data plane free of consensus entirely.

Step ⑤ compares the checksum computed over the received bytes with the one the client sent. A mismatch returns 400 BadDigest without committing anything, which is what makes the checksum end to end rather than decorative. Step ⑥ is the commit: a conditional write of the new version record, carrying the extent pointer from ④, to the partition that owns the key. It is the only step that changes what any reader can see.

A GET runs the same machinery in reverse, with no commit. The front end reads the current version record from the key's partition, fresh on every request (§8 explains why it is never cached), resolves the extent to drive locations from its cache, reads the byte range from the drive holding the data, verifies block checksums, and streams. A large object's chunks are fetched in parallel from different extents, so a 5 GB GET is limited by the client's bandwidth rather than one drive's.

There are four kinds of thing to store: buckets, object versions, in-progress multipart uploads, and extents. They are used in very different ways, by very different parts of the system, at rates that differ by six orders of magnitude, and those differences decide the schema before any field is chosen.

Two observations force the schema. First, LIST is a range scan in key order, so object records must be stored sorted by bucket and key, and partitioned by key range rather than by a hash of the key. Hashing would spread photos/2026/09/ across every partition and turn each LIST page into a scatter-gather over thousands of them. The cost of ordering is that adjacent keys share a partition, and adjacent keys are exactly what a date-prefixed workload hammers; §9 handles that.

Second, bytes move far more often than objects change. Even at half the peak PUT rate, about four million extents a day seal and get coded, each holding around a thousand 1 MB objects. If object records stored drive locations, coding alone would add some 50,000 index writes a second, and every drive failure would rewrite 160,000 fragments' worth of records. So records point at extents, and only the extent map knows where extents are. That one level of indirection keeps every background process in §9 out of the index.

-- Object index: range-partitioned by (bucket_id, key); ~10,000 partitions, 3 replicas each
object_version (
  bucket_id       BIGINT,
  key             BYTES,          -- up to 1,024 bytes, compared bytewise for LIST order
  version_ts      BIGINT DESC,    -- commit timestamp; newest sorts first
  version_id      STRING,         -- opaque id returned to clients
  is_delete_marker BOOL,
  size            BIGINT,
  etag            STRING,
  checksum        BYTES,          -- client-visible, e.g. CRC32C of the whole object
  storage_class   ENUM,           -- STANDARD, INFREQUENT, COLD (offline archive tiers are out of scope)
  chunks          LIST<(extent_id BIGINT, offset BIGINT, length BIGINT)>,
  encryption      (kms_key_id, wrapped_data_key),
  user_metadata   MAP<STRING, STRING>,   -- bounded, ~2 KB
  PRIMARY KEY (bucket_id, key, version_ts DESC)
)

-- Extent map: owned by the extent manager, hash-partitioned by extent_id
extent (
  extent_id   BIGINT PRIMARY KEY,
  state       ENUM,              -- OPEN, SEALED, CODED
  length      BIGINT,
  scheme      ENUM,              -- REPLICA_3, RS_10_8
  epoch       INT,               -- bumped on every move; caches compare it
  locations   LIST<(fragment_idx, drive_id, checksum)>
)
drive_extents (drive_id, extent_id)  -- secondary index: what to rebuild when a drive dies

-- Multipart: separate so parts are invisible to GET and LIST
upload (bucket_id, key, upload_id, created_at, parts MAP<int, (etag, size, extent_id, offset)>)

-- Buckets: small, globally unique names, read on every request and cached
bucket (name PRIMARY KEY, owner, versioning, policy, lifecycle_rules, created_at)

A GET does three lookups in a row: the object record, the extent's location, then the bytes. They look alike, but they change at completely different rates, and how safe each one is to cache depends entirely on how often it changes. The record changes on every PUT to its key. An extent's location changes when background work moves fragments, and it carries an epoch. The bytes at a given extent, offset and length never change at all.

Reading the record fresh is cheap because the partition leader holds a lease, so a linearizable read is a lookup on the leader, not a consensus round. The cost is one extra hop, about a millisecond when the leader is in another zone, plus the lookup itself: 1–5 ms in the GET budget in §6. That budget has room for it. What the design avoids is the harder problem that comes with a cached record, which is invalidating it on a thousand front ends within the time it takes a client to send its next request.

Aggregate load has headroom (§3); the byte cache exists for the single hot object. A 1 MB object that has been coded lives in one fragment on one hard drive, and a hard drive serves somewhere around 100 random reads a second. The request-rate NFR promises 5,500 reads a second per prefix, and nothing stops a customer from sending all of them to a single key: a release artifact, a popular image, a config file every container fetches at startup. Without a cache, that key is capped at about 1/50th of the documented rate by a single drive.

So each front end keeps hot chunks in memory, with a local SSD behind it. Peak egress is 0.9 M reads of 1 MB a second, 900 GB/s, and at around 1 GB/s of useful throughput per front end that is on the order of a thousand front ends. Each one caches independently. A key that suddenly gets popular costs at most one drive read per front end to warm, a thousand reads spread over a few seconds, and after that the drive sees nothing. Because entries are immutable, the cache needs no invalidation, no coherence protocol, and no coordination between front ends. Chunks larger than a few megabytes bypass it; those reads are sequential, spread across many extents, and bound by bandwidth rather than seeks.

§3 showed that aggregate capacity is not what breaks. Four things break instead, either in one place or slowly over time: a key range that runs hot, a drive that runs hot, the constant stream of repairs, and space that deletes leave behind. Each has its own mechanism below. The region-wide layout they operate on comes first.

Hot key ranges: splitting the index

A partition splits for two reasons: size, at around 50 GB, and load. A load split picks the key where the partition's recent traffic divides in half and hands the upper range to a new consensus group. Splits are how a single bucket reaches millions of requests a second: each new range serves its own 3,500 writes and 5,500 reads a second. Traffic is not redirected at once, which is why §5b's 503 SlowDown exists. Clients that back off and retry land on the new partition once the split completes.

Splitting has a blind spot that key ordering creates. If keys are written in increasing order, such as logs/2026-09-17T10:42:01Z-..., every new write lands at the end of the key space, and however the range is split, the last partition takes all of them. A monotonic key sequence therefore caps at one partition's write rate. The fix is on the key, not in the store: put a high-cardinality component, such as a shard number or a hash, before the timestamp so writes spread across ranges. The store's part is to make that limit explicit, per prefix, rather than let it show up as mysterious latency.

Hot drives: placing by heat and free space

The naive placement rule, which sends new extents to the emptiest drives, fails in a way that looks harmless: a freshly installed rack is the emptiest thing in the region, so it receives nearly every new extent, and new data is the data being read. The rack runs hot while older racks idle. Placement therefore weighs recent IOPS as well as free space, and caps the share of new writes any drive can take. New drives fill mostly through the rebalancer instead, which moves cold fragments onto them. Every move is a copy plus one extent-map update (§7); no object record changes.

Repair: declustered and prioritised

At 6.2 drive failures a day (§3), repair never stops. Rebuilding a dead drive onto one spare is the wrong model. Writing 16 TB at around 200 MB/s takes about 22 hours, and reading ten times that from its stripes' survivors would pile onto a handful of drives. Instead, each of the dead drive's ~160,000 fragments is rebuilt independently: a worker reads ten surviving fragments of that stripe, decodes the missing one, and writes it to a drive that respects the zone and rack rule. Thousands of drives each do a sliver. Meeting the 6-hour window from §3 takes 16 TB ÷ 6 h, about 0.74 GB/s written and 7.4 GB/s read per failure, well under a percent of the fleet's bandwidth.

Not every missing fragment is equally urgent. A stripe missing one fragment still has seven spare; a stripe missing four is halfway to its margin. The repair queue is ordered by fragments missing per stripe, and a repair budget, a capped share of each drive's IOPS, keeps rebuilds from starving customer reads. When stripes get close to their margin, the budget is lifted. Alongside repair, a scrubber reads and verifies every fragment's checksums on a fixed cycle, so latent sector errors are found while the stripe can still rebuild them. A two-week cycle over 1.8 EB is about 13 MB/s per drive, 5–7% of a drive's 200–250 MB/s of sequential bandwidth.

Reclaiming space: compaction

A DELETE or an overwrite changes only the index, and the old bytes stay where they are inside a sealed, coded extent. The extent manager keeps an approximate garbage count per extent, fed by deletes. Once an extent crosses a threshold, a compactor rewrites it. It reads the list of (key, version, offset) that every extent keeps in its footer, checks each entry against the object index and the multipart upload table, since a part of an open upload is live but has no object record yet, and copies the live bytes into a new extent. It then moves each record with a conditional update, "only if this record still points at extent E, offset O", because the key may have been overwritten while the copy ran. Compaction is the one background process that writes to the object index, so it is throttled.

The threshold trades space for rewrite work. Compacting an extent that is g garbage reclaims g and copies 1 − g, so at 50% garbage every byte reclaimed costs one byte copied, and at 20% it costs four. The old extent is not deleted when the copy finishes. It is scheduled for deletion days later, because a bug in the liveness check would otherwise become permanent data loss within the hour (§10).

Hardware failure is the part of this system that is already designed for: a drive dies every few hours, and §5 and §9 turn that into routine work. The failures worth discussing are the ones where the routine machinery is the wrong response, such as rebuilding a zone that will be back in an hour, and the ones that redundancy cannot help with at all, because the store's own software did the damage and then faithfully replicated it.

A zone outage, in numbers

Reads. On average a third of all data fragments sit in the lost zone, so a third of reads become degraded reads of ten fragments each. At peak, 0.9 M reads a second become 0.3 M × 10 + 0.6 M = 3.6 M, which is 4× the load. If the lost zone happens to hold four of a stripe's ten data fragments, as in Figure 2's layout, the figure is 4.1 M, or 4.6×. The surviving 75,000 drives manage about 7.5 M random reads a second between them, so the outage takes them to roughly half their random-read capacity. That is survivable because in normal operation the fleet runs at under a tenth of its random-read capacity (§3); even at 95% full, with fewer drives, the outage would reach only about 60%. The hot-chunk cache (§8) matters more during an outage than at any other time.

Writes. An open extent needs three replicas, and only two zones are left. New extents are placed two plus one across the surviving zones, which still survives the loss of either zone. Coding pauses, because 18 fragments over two zones means 9 per zone, one more than the code can lose. New data stays replicated at 3× until the zone is back. At an average ingest of 50 GB/s that costs about 5 PB of extra raw storage per day of outage, small against the free space the 80% fill leaves.

Repair. The routine repair loop would treat the outage as 37,500 dead drives and try to rebuild 600 PB into two zones that cannot legally hold it. So repair distinguishes unavailable from lost. A zone that is unreachable is assumed to come back, and only stripes that also lose fragments in the surviving zones get repaired, temporarily breaking the zone rule if needed, since a stripe with one spare is in more danger than a stripe that is lopsided. Declaring a zone permanently lost is left to a human, because its cost is measured in days of degraded capacity.

Object storage is where an organisation's data ends up by default. Backups, logs, datasets and user uploads all land there, often without anyone deciding who should be able to read them. Most incidents involving object stores are therefore access failures: data that was readable when it should not have been, or deleted by someone who should not have been able to. Exposure is handled by defaults: account- and bucket-level public-access blocks override any policy that tries to grant public access, and policy evaluation denies unless something explicitly allows it.

Ransomware and malicious deletion

An attacker holding stolen credentials with delete permission can do in minutes what no hardware failure can, and the store will replicate the deletions to all three zones. Versioning helps only partly: a DELETE becomes a delete marker, and older versions survive, until the attacker deletes those as well. Object Lock (§5b) closes the gap. In compliance mode, a version under retention cannot be deleted or overwritten by anyone, including the account root, until its retention date passes. Enforcement lives in the index partition: the delete path checks the retention field in the version record before committing a removal, so no credential can bypass it. That makes it the answer to ransomware and to regulations that require write-once storage.

Encryption, and deletion that has to be real

Since January 2023, S3 encrypts every new object by default. The usual design is envelope encryption: each object gets its own data key, the bytes are encrypted with it, and the data key is stored in the object record wrapped by a key in a key management service. A customer who revokes access to that outer key makes their data unreadable immediately, without the store moving a byte.

This matters because §10 makes deletion deliberately slow, while privacy law requires erasure to be real. The two fit together. When a version is deleted, its record, and the wrapped data key with it, leaves the index at once. What lingers for the grace period is ciphertext in unreferenced extents, plus the index change log that also expires within that window. Erasure is therefore complete within a bounded, documented time, and anything recovered from the grace window is recovered deliberately, not by accident.

Nearly every candidate draws the same boxes: a metadata service, a fleet of storage nodes, and a front end between them. The levels separate on what the candidate can say about those boxes. The biggest jump is from L5 to L6, where the candidate derives eleven nines instead of stating it, and derives the parity count from the zone rule.

Classic probes, and how the answers differ by level

Latency: the numbers that set the budget