Suppose an interviewer asks: \"How long does it take to read 1 GB from an SSD?\" You don't need a benchmark to answer. If you remember that reading 1 MB in sequence from an SSD takes about 1 ms, then 1 GB, which is 1,000 MB, takes about 1,000 ms, or one second. The same read from a spinning hard disk takes about 20 seconds, and from main memory about a quarter of a second. Those three answers already tell you a lot about where a design should keep its hot data.
\nEstimates like this come up all the time in design work. You might need to know how long it takes to generate 100 image thumbnails from disk, how much memory a data structure will take, or whether a link is fast enough to copy a backup overnight. Each of these needs only a few numbers that you know by heart, plus some careful arithmetic. This chapter collects those numbers in one place: powers of two, the classic latency table, throughput figures, and a few numbers for AI systems. The method for using them, step by step, is in Back-of-the-envelope estimation.
\nA word on age. The famous latency table dates from around 2009 to 2012. Some of its rows have hardly moved since then, and others have improved more than tenfold. This chapter keeps the classic table as it was, because people still quote it, and then shows a modern ballpark next to it, with a source and a year for each number.
","headings":[]},{"id":"powers-of-two","title":"Powers of two","html":"Computers count in binary, so sizes and limits tend to land on powers of two. A few of them are worth knowing on sight.
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Power | Exact value | Approximate value | Bytes |
|---|---|---|---|
| 2^7 | 128 | ||
| 2^8 | 256 | ||
| 2^10 | 1,024 | 1 thousand | 1 KB |
| 2^16 | 65,536 | 64 KB | |
| 2^20 | 1,048,576 | 1 million | 1 MB |
| 2^30 | 1,073,741,824 | 1 billion | 1 GB |
| 2^32 | 4,294,967,296 | 4 GB | |
| 2^40 | 1,099,511,627,776 | 1 trillion | 1 TB |
| 2^50 | 1,125,899,906,842,624 | 1 quadrillion | 1 PB |
Read the table like this. Each step of 10 in the power multiplies the value by 1,024, which is close to 1,000. So 2^10 is about a thousand, 2^20 about a million, 2^30 about a billion and 2^40 about a trillion.
\nTwo rows are worth remembering for their own sake:
\nThe \"Bytes\" column above uses the same names for two different things. In everyday use, a kilobyte (KB) can mean either 1,000 bytes or 1,024 bytes, and a gigabyte can mean either 10^9 bytes or 2^30 bytes.
\nTo remove the confusion, the International Electrotechnical Commission (IEC) approved separate names for the binary sizes in December 1998, according to NIST's page on binary prefixes:
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Decimal (SI) unit | Size | Binary (IEC) unit | Size |
|---|---|---|---|
| kilobyte, kB | 10^3 = 1,000 bytes | kibibyte, KiB | 2^10 = 1,024 bytes |
| megabyte, MB | 10^6 = 1,000,000 bytes | mebibyte, MiB | 2^20 = 1,048,576 bytes |
| gigabyte, GB | 10^9 bytes | gibibyte, GiB | 2^30 = 1,073,741,824 bytes |
| terabyte, TB | 10^12 bytes | tebibyte, TiB | 2^40 bytes |
The gap between the two grows with size. A KiB is 2.4% bigger than a kB. A MiB is 4.9% bigger than an MB, a GiB 7.4% bigger than a GB, a TiB 10% bigger than a TB, and a PiB about 12.6% bigger than a PB.
\nWhen does it matter?
\nThese are the conversions people use most in estimates. The \"shortcut\" column is the rounded number you do arithmetic with.
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Quantity | Exact or close | Shortcut |
|---|---|---|
| Seconds in a day | 86,400 | about 10^5 |
| Seconds in a 30-day month | 2,592,000 | about 2.6 million, or 2.5 × 10^6 |
| Seconds in a year | 31,536,000 | about 3 × 10^7 |
| 1 million requests a day | 11.6 a second | about 12 a second |
| 1 billion requests a day | 11,574 a second | about 10^4 a second |
| 1 Gbps (gigabit per second) | 125 MB/s | about 100 MB/s after protocol overhead |
| Bits to bytes | divide by 8 |
Two quick checks show how the shortcuts work. If a service gets 100 million requests a day, divide by 10^5 to get about 1,000 requests a second on average. If a server writes 1 KB per request at 1,000 requests a second, that is 1 MB a second, or about 86 GB a day.
","headings":[{"id":"powers-of-two","title":"Powers of two","depth":3},{"id":"decimal-and-binary-units","title":"Decimal and binary units","depth":4},{"id":"quick-conversions","title":"Quick conversions","depth":4}]},{"id":"latency-numbers-every-programmer-should-know","title":"Latency numbers every programmer should know","html":"Latency is how long one operation takes from start to finish. The table below lists the latency of common operations, from reading a CPU cache to sending a packet across an ocean. The list was first compiled by Peter Norvig, in the answers section of his essay Teach Yourself Programming in Ten Years. Jeff Dean of Google then updated it and showed it in his talks, under the title \"Numbers Everyone Should Know\". Jonas Bonér's gist of the table, labelled \"~2012\", is the version most people copy.
\nUnits: 1 ns (nanosecond) = 10^-9 seconds. 1 µs (microsecond) = 10^-6 seconds = 1,000 ns. 1 ms (millisecond) = 10^-3 seconds = 1,000 µs = 1,000,000 ns.
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Operation | ns | µs | ms | Notes |
|---|---|---|---|---|
| L1 cache reference | 0.5 | |||
| Branch mispredict | 5 | |||
| L2 cache reference | 7 | 14 × L1 cache | ||
| Mutex lock/unlock | 25 | |||
| Main memory reference | 100 | 14 × L2 cache, 200 × L1 cache | ||
| Compress 1 KB with Zippy | 3,000 | 3 | ||
| Send 1 KB over a 1 Gbps network | 10,000 | 10 | ||
| Read 4 KB randomly from SSD | 150,000 | 150 | assumes an SSD that reads about 1 GB/s | |
| Read 1 MB sequentially from memory | 250,000 | 250 | ||
| Round trip within the same data center | 500,000 | 500 | 0.5 | |
| Read 1 MB sequentially from SSD | 1,000,000 | 1,000 | 1 | about 1 GB/s SSD; 4 × memory |
| HDD seek | 10,000,000 | 10,000 | 10 | 20 × data center round trip |
| Read 1 MB sequentially over a 1 Gbps network | 10,000,000 | 10,000 | 10 | 40 × memory, 10 × SSD |
| Read 1 MB sequentially from HDD | 20,000,000 | 20,000 | 20 | 80 × memory, 20 × SSD |
| Send a packet California → Netherlands → California | 150,000,000 | 150,000 | 150 |
A few terms, in case they are new:
\nif will go, starts work on that path, and has to throw the work away because it guessed wrong.Different versions of the table disagree on a few rows. The version above follows Jeff Dean's slide from his October 2009 LADIS keynote, Designs, Lessons and Advice from Building Large Distributed Systems, and the jboner gist. An older slide from Dean's Stanford talk, Software Engineering Advice from Building Large-Scale Distributed Systems, gave slower numbers: 100 ns for a mutex, 10 µs to compress 1 KB with Zippy, and 30 ms to read 1 MB from disk. The row for reading 1 MB over a 1 Gbps network comes from that older slide. It checks out: 1 MB is 8 million bits, which takes 8 ms at 1 Gbps, so \"about 10 ms\" is fair.
\nThe table was also drawn as a picture of grids of squares, one square per unit of time, which makes the jumps between rows easy to see. Colin Scott's interactive version adds a year slider, covered in How these numbers have changed below.
\nThe exact values matter less than the gaps between them. Here is what the gaps say, in plain sentences.
\nNanoseconds are hard to picture. One way to feel the gaps is to stretch time so that an L1 cache reference takes one second. That multiplies every number by 2 billion.
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Operation | Real time | If an L1 cache reference took 1 second |
|---|---|---|
| L1 cache reference | 0.5 ns | 1 second |
| Branch mispredict | 5 ns | 10 seconds |
| L2 cache reference | 7 ns | 14 seconds |
| Mutex lock/unlock | 25 ns | 50 seconds |
| Main memory reference | 100 ns | 3.3 minutes |
| Compress 1 KB with Zippy | 3 µs | 1.7 hours |
| Send 1 KB over a 1 Gbps network | 10 µs | 5.6 hours |
| Read 4 KB randomly from SSD | 150 µs | 3.5 days |
| Read 1 MB sequentially from memory | 250 µs | 5.8 days |
| Round trip within the same data center | 500 µs | 11.6 days |
| Read 1 MB sequentially from SSD | 1 ms | 23 days |
| HDD seek | 10 ms | 7.6 months |
| Read 1 MB sequentially over a 1 Gbps network | 10 ms | 7.6 months |
| Read 1 MB sequentially from HDD | 20 ms | 15 months |
| California → Netherlands → California | 150 ms | 9.5 years |
On this scale, fetching something from memory is a short coffee break. A round trip inside the data center is a two-week holiday. A round trip across the Atlantic takes most of a decade.
\nEach rate below comes from one row of the table. Divide the size by the time.
\nThe last two describe one caller waiting for each reply before sending the next request. A server talking to many callers at once can do far more round trips in total.
\nHardware has moved on since the table was written, but unevenly. Colin Scott's interactive latency page shows this by letting you pick a year. It starts from Norvig's numbers and applies a trend to each row (source code on GitHub). In its model, CPU clock speed stops rising in 2005, main memory latency stays at 100 ns from 2000 on, the data-center round trip stays at 500 µs, and the California–Netherlands round trip stays at 150 ms because it is limited by the speed of light. Storage and network bandwidth keep growing. Treat the page as a model of trends. Its numbers are extrapolations from older data.
\nFor a few rows that have moved the most, here is a modern ballpark from current vendor specifications and published measurements. The classic table above stays as it is; use these when you want a number closer to today's hardware.
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Row | Classic table | A modern ballpark | Source and year |
|---|---|---|---|
| Read 4 KB randomly from SSD | 150 µs | about 60 µs typical read latency for a data-center NVMe SSD | Solidigm D7-PS1010 spec table, in StorageReview's review, 2024 |
| Read 1 MB sequentially from SSD | 1 ms (about 1 GB/s) | about 70 µs, at up to 14,500 MB/s | Solidigm D7-PS1010 product page, a PCIe 5.0 drive from 2024 |
| Round trip within the same data center | 500 µs | about 50 to 150 µs between two VMs in one cloud zone (netperf mean 67 µs; ping average 146 µs) | Google Cloud blog, What a trip! Measuring network latency in the cloud, 2020 |
| Read 1 MB sequentially from memory | 250 µs (about 4 GB/s) | a 5th Gen AMD EPYC server CPU has a peak memory bandwidth of 614 GB/s per socket, shared by all its cores | AMD EPYC 9575F spec page |
| HDD seek, and 1 MB from HDD | 10 ms and 20 ms | average rotational latency 4.16 ms; up to 285 MB/s sustained, so 1 MB in about 3.5 ms; about 170 random 4 KB reads a second | Seagate Exos data sheet (ds2046.1), current in 2026 |
| Main memory reference | 100 ns | still about 100 ns | Colin Scott's model holds it flat |
| California → Netherlands → California | 150 ms | still about 150 ms | limited by distance |
Some notes on reading this table:
\nThroughput is how much work a system finishes per unit of time: bytes per second for a disk, requests per second for a server. The latency table hides throughput inside its rows. This section pulls it out.
\n| Medium | Rough rate | Time to move 1 GB | Where the number comes from |
|---|---|---|---|
| Main memory | 4 GB/s in the classic table; up to 614 GB/s peak for a whole modern server socket | 0.25 s classic | Classic table; AMD EPYC spec page |
| NVMe SSD | 1 GB/s classic; up to 14.5 GB/s for a 2024 PCIe 5.0 data-center drive | 1 s classic; about 0.07 s modern | Classic table; Solidigm D7-PS1010 |
| Hard disk | 50 MB/s classic; up to 285 MB/s for a current large drive | 20 s classic; about 3.5 s modern | Classic table; Seagate Exos data sheet |
| 1 Gbps network | 125 MB/s line rate; plan on about 100 MB/s | about 10 s | arithmetic: 1 Gbps ÷ 8 bits per byte |
| 10 Gbps network | 1.25 GB/s | about 1 s | arithmetic |
| 100 Gbps network | 12.5 GB/s | about 0.1 s | arithmetic |
A rule of thumb follows from this table. A fast modern SSD reads data about as fast as a 100 Gbps network can carry it, and a hard disk reads about as fast as a 2 Gbps link. So in a data pipeline, the network card often runs out of capacity before the drive does.
\nHow many requests one machine can handle depends heavily on what each request does. The numbers below are rough orders of magnitude for sizing. Real systems often do much less, so measure your own before you commit to a design.
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| Server type | Rough throughput | Basis |
|---|---|---|
| In-memory cache node (Redis) | about 70,000 to 180,000 simple commands a second without pipelining; over 1.5 million with 16 commands per pipeline | Examples in the Redis benchmark docs. Redis runs commands on mostly one thread, so one instance uses about one core. |
| Relational database node, simple reads | about 530,000 to 840,000 primary-key lookups a second in a synthetic test | PostgreSQL's pgbench -S (select-only) on a 16-core, 32-thread desktop CPU, with the client on the same machine and all data in memory, reported on the pgsql-hackers list in 2023 |
| Relational database node, real workload | assumption: about 1,000 to 10,000s of queries a second | Real queries join tables, write to disk and wait on the network and on locks. This range is a planning assumption. |
| Web or app server, dynamic pages | assumption: hundreds to a few thousand requests a second | See the derivation below. |
The app server row is easiest to reason about from first principles. If each request needs about 10 ms of CPU time and the server has 16 cores, the most it can do is 16 × (1,000 ms ÷ 10 ms) = 1,600 requests a second. In practice you plan to run well below that peak, because waiting times rise sharply as a server gets close to fully busy (see Performance and latency). A server that mostly waits on other services, rather than computing, can hold many more requests at once, and then memory or connection limits set the ceiling instead.
\nThe gap between the synthetic database result and a real workload is the main lesson of this table. Benchmarks measure the best case. A design should assume much less, and check against measurements once the system exists.
","headings":[{"id":"throughput-numbers","title":"Throughput numbers","depth":3},{"id":"sequential-read-and-transfer-rates","title":"Sequential read and transfer rates","depth":4},{"id":"requests-per-second-for-one-machine","title":"Requests per second for one machine","depth":4}]},{"id":"numbers-for-ai-systems","title":"Numbers for AI systems","html":"AI models add a new kind of estimate: will the model fit in memory, and how fast can it run? The basic numbers come from the GPU's specification and from the model's size.
\nMemory and bandwidth of one data-center GPU. NVIDIA's spec pages list:
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n| GPU | Memory (HBM) | Memory bandwidth | Source |
|---|---|---|---|
| H100 SXM | 80 GB | 3.35 TB/s | NVIDIA H100 |
| H100 NVL | 94 GB | 3.9 TB/s | same page |
| H200 SXM | 141 GB | 4.8 TB/s | NVIDIA H200 |
HBM (high-bandwidth memory) is the memory stacked next to the GPU chip. Its bandwidth is several times higher than a whole server CPU socket's memory bandwidth (614 GB/s above).
\nHow big a model's weights are. A model's weights take up roughly:
\nweight size = number of parameters × bytes per parameterThe bytes per parameter depend on the number format. FP16 and BF16 (16-bit floating point) use 2 bytes. 8-bit formats use 1 byte. 4-bit formats use half a byte. So:
\nThis is only the weights. Running a model also needs memory for intermediate results, mainly the cache of each conversation's past tokens. So a 140 GB model does not fit on one 80 GB H100, and it only just fits within one H200's 141 GB with almost no room left over. In practice, it would run on two or more GPUs.
\nA rough speed limit. When a model generates text one token at a time for a single user, each new token needs the GPU to read all the weights from memory once. Reading 140 GB at 4.8 TB/s takes about 29 ms. So one H200 can produce at most about 34 tokens a second for that single conversation, however fast its arithmetic units are. Serving many users at once in the same pass over the weights is how real systems get much higher total throughput.
\nThe gist that popularized the latency table added rows for language models in 2026. They are rough, but useful for scale: a hosted frontier model produces a token about every 20 ms, takes about 1 second before its first token on a short prompt, about 10 seconds to read a 100,000-token prompt with no cache, and about 30 seconds for a single reasoning response.
\nA later part of the handbook covers model inference in depth: batching, caching, and how to size a serving fleet.
","headings":[{"id":"numbers-for-ai-systems","title":"Numbers for AI systems","depth":3}]},{"id":"check-yourself","title":"Check yourself","html":"Work from the 1 MB rows and multiply by 1,000.
\nWith a 2024 PCIe 5.0 SSD at 14.5 GB/s, the SSD read drops to about 0.07 seconds, faster than the classic table's memory figure.
\nEach read needs one seek (10 ms) plus 0.256 MB at 20 ms per MB (about 5 ms). That is about 15 ms per thumbnail.
\nIf the thumbnails sit in a memory cache instead, reading all 25.6 MB takes about 25.6 × 250 µs ≈ 6.4 ms. Jeff Dean's 2009 slides used the same kind of estimate for a page of 30 thumbnails: about 560 ms read one at a time, about 18 ms read in parallel.
\nNo. 2^32 = 4,294,967,296, about 4.3 billion, which is fewer than 6 billion. A signed 32-bit integer only reaches about 2.1 billion. Use a 64-bit ID, which holds about 1.8 × 10^19 values. See Unique IDs for ways to generate them.
\n10 TB = 10 × 10^12 bytes × 8 = 8 × 10^13 bits.
\nBoth assume the link runs at full speed the whole time, which real transfers rarely do. Add some margin.
\nEach round trip takes about 150 ms, so 4 × 150 ms = 600 ms before any server work. Faster servers can't fix this, because the time goes on distance. Serve the content from a location near the user with a CDN or a nearby data center, and cut the number of round trips, for example by reusing connections and combining requests.
\nIn FP16 the weights alone take 70 × 10^9 × 2 bytes = 140 GB, which is far more than 80 GB, so it needs at least two GPUs. In an 8-bit format the weights take about 70 GB. That fits, but it leaves only about 10 GB for the per-conversation cache and other working memory, which limits how many users it can serve at once.
\nThe classic sources for the latency table:
\nUnits and modern figures:
\n