Real-world architectures

The rest of this handbook teaches ideas one at a time: caching, replication, sharding, queues, consensus. Real systems use many of them at once, and the interesting part is how they fit together and what each team gave up to get what it needed. Reading how real systems were built is the best follow-up to the handbook, for three reasons:

• You see the trade-offs being made for real. A chapter can tell you that a store can favor availability over consistency. A paper shows you a team that did it, why, and what broke afterwards.
• You collect patterns. After a few papers you notice the same small set of techniques again and again: a ring of hashed keys, a single master that holds metadata, a log that everything writes to, a cache in front of the database.
• Interview questions often come from the same domain. If you are interviewing at a company, its engineering blog shows the problems its engineers care about and the words they use for them.

This chapter is a reading list with notes. It has three parts: the papers and talks behind well-known systems, write-ups of how particular companies built their products, and the engineering blogs where companies still publish this kind of material.

Details such as version numbers, machine counts and product names go out of date quickly, so there is no point memorizing them. Read for the shape of the design, with four questions in mind:

• What problem were they solving, and for whom?
• What constraints shaped the design: the size of the data, the read-to-write ratio, the latency target, the size of the team?
• Which trade-offs did they make, and what did they give up?
• What would they change, or what did they change later?

A worked example: reading the Dynamo paper

Here is what that looks like on one paper. Dynamo: Amazon's Highly Available Key-value Store was presented at SOSP in October 2007. It describes the store that some of Amazon's core services used to keep data such as shopping carts.

The problem. The paper opens with the shopping cart. Customers "should be able to view and add items to their shopping cart even if disks are failing, network routes are flapping, or data centers are being destroyed by tornados." At Amazon's size, the authors write, small and large components fail continuously. So the store must keep accepting writes while parts of it are broken.

The constraints. Many Amazon services, such as shopping carts, best seller lists, session management, sales rank and the product catalog, only need to look data up by its primary key. The paper says that storing their data in a relational database "would lead to inefficiencies and limit scale and availability," so Dynamo offers only a primary-key interface. Amazon also measured its service-level agreements at the 99.9th percentile of latency, so that the slowest 0.1% of requests counted. The paper reports that its 99.9th-percentile latencies were around 200 ms, an order of magnitude higher than the averages.

The trade-offs. To stay writable during failures, "Dynamo sacrifices consistency under certain failure scenarios." Two copies of the same cart can be updated separately and drift apart. Dynamo keeps both versions and lets the application merge them when it next reads the data. The design pushes "the complexity of conflict resolution to the reads in order to ensure that writes are never rejected." The cost shows up in the cart: an "add to cart" operation is never lost, "however, deleted items can resurface."

The techniques. The paper's Table 1 maps each problem to the technique it used. Every one of them has a section in this handbook:

The paper also shows the tuning knobs from quorums: each key is stored on N machines, a read waits for R of them and a write waits for W. "The common (N,R,W) configuration used by several instances of Dynamo is (3,2,2)." Because R + W = 4 is greater than N = 3, a read and a write that both succeed normally overlap on at least one machine.

What you take away. You don't need to remember any of Dynamo's numbers. The lesson is the pattern: when losing a write costs more than showing slightly stale data, accept every write and merge conflicts later. The CAP chapter tells the same story from the theory side.

What changed later. Amazon's managed database service, DynamoDB, has a similar name but is a different system. Its own paper, Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service, appeared at USENIX ATC in July 2022. Reading the two papers side by side is a good exercise in how a design changes when it becomes a service that outside customers pay for.

How to read an architecture write-up

Use this checklist on any paper, talk or blog post in this chapter.

1. Find the problem first. Read the abstract and the introduction, then stop and write one sentence: who needed what, and why the existing tools weren't good enough.
2. Write down the numbers that drove the design. Requests per second, reads versus writes, data size, latency targets, team size. These explain most of the choices.
3. Sketch the architecture. Draw the boxes and arrows yourself before you look at the paper's figure. Mark where the data lives and which parts can fail on their own.
4. For each component, ask what it solves. Then ask where it works, where it doesn't, and what would break first if traffic grew ten times.
5. Name the trade-offs. Look for what they gave up: consistency for availability, flexibility for speed, generality for simplicity. Most papers say this outright in one or two sentences.
6. Look for shared patterns and technologies. Compare it with other systems you have read about. Most designs reuse a small set of ideas, and spotting them is the real skill.
7. Read the lessons learned. The "experience" or "lessons" section near the end is often the most useful part, because it is where the authors say what surprised them.
8. Check the date. A 2008 post about a site running on a few dozen servers describes a different world from today's managed cloud services. The reasoning can still hold even when the products have changed.
9. Skip the nitty-gritty on a first pass. Proofs, benchmark setups and configuration details can wait until you need them.

Systems and papers

These are the systems most often cited in system design discussions. Where the original paper is freely available, it's linked. For open source projects, the link is an introductory slide deck, with the project's own site or original paper alongside it. The one-sentence descriptions come from the paper or project page.

A worked example of the kind of diagram these write-ups contain: in a 2013 talk, Timelines at Scale, Twitter's Raffi Krikorian walked through how a tweet reaches people. High Scalability's summary of the talk reports 300,000 read requests a second against 6,000 writes, so Twitter did the work when a tweet was written. A "fanout" step copied each new tweet's ID into every follower's home timeline, kept in a Redis cluster with up to 800 entries per timeline and three copies of each. A separate path fed the tweet to an "Ingester", which indexed it in Earlybird, Twitter's search index, and a service called Blender built search results from it. The tweet also triggered push notifications.

Fan-out on write has a weak spot: accounts with tens of millions of followers. The summary reports that a tweet could take up to five minutes to reach all 31 million of Lady Gaga's followers, against a goal of five seconds. For accounts like that, Twitter was moving to skip the fan-out and merge their tweets into each reader's timeline at read time. Design the Twitter timeline and search builds the same design step by step.

Data processing

Data stores

File systems

Coordination and other infrastructure

Each paper describes its system as it was when the paper was published, often more than a decade ago. The systems have changed since, and some have been replaced. Read the papers for the reasoning behind each design, which ages much more slowly than the details.

Company architectures

These write-ups describe how particular companies built their products. Most are posts on High Scalability, a blog that summarized conference talks and engineering posts for many years. Many are more than ten years old, so the year matters: read them for how the team reasoned under its constraints, and assume the specific tools have changed since. The year is the post's publication date, or the date of the talk where that's known.

Some pairs are worth reading together, because each one shows a design changing as the company grew:

• Facebook's three papers cover one company's storage at three layers. Memcache is a cache in front of MySQL. TAO is a graph store that replaced memcache for many kinds of data (compare it with Design the data structures for a social network). Haystack is a photo store that keeps each photo's metadata in memory, so that reading a photo costs fewer disk operations.
• The two Pinterest posts, from 2012 and 2013, cover the same team a year apart.
• The seven Twitter entries run from 2009 to 2016 and follow one company as it grew. Read them with Design the Twitter timeline and search.

Company engineering blogs

If you are interviewing at a company, read a few recent posts from its engineering blog. The questions you get may come from the same domain, and the posts show which problems the company's engineers actually work on.

Many of these blogs have moved since they were first started, and a few have closed. The links below are the current addresses, checked in October 2026. Where a blog has closed, the link is to an archived copy on the Internet Archive's Wayback Machine.

For many more, the community-maintained engineering-blogs list on GitHub collects hundreds of company and personal engineering blogs. If you know a company blog that's missing, add it there.

Further reading

• The Tail at Scale: Dean and Barroso on why large systems care about their slowest requests, the same concern that made Amazon measure Dynamo at the 99.9th percentile. Performance and latency walks through it.
• kilimchoi/engineering-blogs: the largest maintained list of engineering blogs.
• High Scalability: the archive behind most of the company write-ups in this chapter, searchable by company name.
• Design a key-value store, Design the Twitter timeline and search and Design the data structures for a social network: the handbook designs that lean most on the papers above.