Numbers to know

Suppose an interviewer asks: "How long does it take to read 1 GB from an SSD?" You don't need a benchmark to answer. If you remember that reading 1 MB in sequence from an SSD takes about 1 ms, then 1 GB, which is 1,000 MB, takes about 1,000 ms, or one second. The same read from a spinning hard disk takes about 20 seconds, and from main memory about a quarter of a second. Those three answers already tell you a lot about where a design should keep its hot data.

Estimates like this come up all the time in design work. You might need to know how long it takes to generate 100 image thumbnails from disk, how much memory a data structure will take, or whether a link is fast enough to copy a backup overnight. Each of these needs only a few numbers that you know by heart, plus some careful arithmetic. This chapter collects those numbers in one place: powers of two, the classic latency table, throughput figures, and a few numbers for AI systems. The method for using them, step by step, is in Back-of-the-envelope estimation.

A word on age. The famous latency table dates from around 2009 to 2012. Some of its rows have hardly moved since then, and others have improved more than tenfold. This chapter keeps the classic table as it was, because people still quote it, and then shows a modern ballpark next to it, with a source and a year for each number.

Powers of two

Computers count in binary, so sizes and limits tend to land on powers of two. A few of them are worth knowing on sight.

Read the table like this. Each step of 10 in the power multiplies the value by 1,024, which is close to 1,000. So 2^10 is about a thousand, 2^20 about a million, 2^30 about a billion and 2^40 about a trillion.

Two rows are worth remembering for their own sake:

• 2^8 = 256 is the number of different values one byte can hold (0 to 255).
• 2^32 ≈ 4.3 billion is the number of different values a 32-bit unsigned integer can hold. If you plan to give every user, photo or event an ID, and you expect more than about 4 billion of them, a 32-bit ID will run out. A 64-bit ID holds about 1.8 × 10^19 values, which is enough for almost anything.

Decimal and binary units

The "Bytes" column above uses the same names for two different things. In everyday use, a kilobyte (KB) can mean either 1,000 bytes or 1,024 bytes, and a gigabyte can mean either 10^9 bytes or 2^30 bytes.

To remove the confusion, the International Electrotechnical Commission (IEC) approved separate names for the binary sizes in December 1998, according to NIST's page on binary prefixes:

The gap between the two grows with size. A KiB is 2.4% bigger than a kB. A MiB is 4.9% bigger than an MB, a GiB 7.4% bigger than a GB, a TiB 10% bigger than a TB, and a PiB about 12.6% bigger than a PB.

When does it matter?

• In estimates, it almost never matters. Your assumptions about users and object sizes are rarely within 10% of the truth. Use powers of ten, because the arithmetic is easier.
• It matters when you compare quoted figures. Disk makers and network vendors use decimal units. Operating systems and memory sizes often use binary ones. For example, a hard-disk datasheet from Seagate lists one drive's top transfer rate as "285 MB/s / 272 MiB/s": the same speed in both units. A "1 TB" disk shows up as about 931 GiB in an operating system that counts in binary, and nothing is missing.
• It matters when you set hard limits. If a field, a buffer or a quota must hold exactly 64 KiB, write 65,536 bytes in the code. 64,000 bytes is about 2% short.

Quick conversions

These are the conversions people use most in estimates. The "shortcut" column is the rounded number you do arithmetic with.

Two quick checks show how the shortcuts work. If a service gets 100 million requests a day, divide by 10^5 to get about 1,000 requests a second on average. If a server writes 1 KB per request at 1,000 requests a second, that is 1 MB a second, or about 86 GB a day.

Latency numbers every programmer should know

Latency is how long one operation takes from start to finish. The table below lists the latency of common operations, from reading a CPU cache to sending a packet across an ocean. The list was first compiled by Peter Norvig, in the answers section of his essay Teach Yourself Programming in Ten Years. Jeff Dean of Google then updated it and showed it in his talks, under the title "Numbers Everyone Should Know". Jonas Bonér's gist of the table, labelled "~2012", is the version most people copy.

Units: 1 ns (nanosecond) = 10^-9 seconds. 1 µs (microsecond) = 10^-6 seconds = 1,000 ns. 1 ms (millisecond) = 10^-3 seconds = 1,000 µs = 1,000,000 ns.

A few terms, in case they are new:

• An L1 cache and an L2 cache are small, fast memories inside the CPU. L1 is the smallest and fastest. Main memory (RAM) sits outside the CPU and is much slower to reach.
• A branch mispredict happens when the CPU guesses which way an if will go, starts work on that path, and has to throw the work away because it guessed wrong.
• A mutex is a lock that lets only one thread at a time into a section of code.
• Zippy was Google's internal name for a fast compression library, released as open source in 2011 under the name Snappy.
• An HDD seek is the time a hard disk takes to move its read head to the right track.
• Sequential means reading bytes that sit next to each other. Random means jumping to a new place for each read.

Different versions of the table disagree on a few rows. The version above follows Jeff Dean's slide from his October 2009 LADIS keynote, Designs, Lessons and Advice from Building Large Distributed Systems, and the jboner gist. An older slide from Dean's Stanford talk, Software Engineering Advice from Building Large-Scale Distributed Systems, gave slower numbers: 100 ns for a mutex, 10 µs to compress 1 KB with Zippy, and 30 ms to read 1 MB from disk. The row for reading 1 MB over a 1 Gbps network comes from that older slide. It checks out: 1 MB is 8 million bits, which takes 8 ms at 1 Gbps, so "about 10 ms" is fair.

The table was also drawn as a picture of grids of squares, one square per unit of time, which makes the jumps between rows easy to see. Colin Scott's interactive version adds a year slider, covered in How these numbers have changed below.

What the table tells you

The exact values matter less than the gaps between them. Here is what the gaps say, in plain sentences.

• Memory is far faster than any disk. A main memory reference takes 100 ns. A random 4 KB read from an SSD takes 150 µs, which is 1,500 times longer. This is the whole case for caching: keep hot data in memory.
• Sequential reads beat random reads. A hard disk spends 10 ms just finding the data, and then reads a whole megabyte in 20 ms. If you read 1 MB as 256 separate 4 KB pieces scattered over the disk, you pay 256 seeks, which is about 2.5 seconds. Databases and logs are built around this: they append to the end of a file and read in large blocks.
• The network inside a data center is faster than a local hard disk. A round trip to another machine in the same building takes 0.5 ms. One disk seek takes 10 ms. So asking another server's memory, for example a cache like memcached or Redis, is quicker than reading your own disk.
• Long-distance round trips are expensive. California to the Netherlands and back takes 150 ms, which is 300 round trips inside a data center. A page that needs four round trips to a faraway server spends 600 ms waiting before any other work. That is why CDNs put content near users, and why APIs used over long distances should avoid many small back-and-forth calls.
• Compression is cheap compared with sending data. Compressing 1 KB takes 3 µs, and sending 1 KB over a 1 Gbps link takes 10 µs. If compression halves the size, it often pays for itself.
• CPU-level costs are tiny next to I/O. A branch mispredict or a cache miss matters inside tight loops. In a request that does one disk read or one network call, they disappear.

A human-scale version

Nanoseconds are hard to picture. One way to feel the gaps is to stretch time so that an L1 cache reference takes one second. That multiplies every number by 2 billion.

On this scale, fetching something from memory is a short coffee break. A round trip inside the data center is a two-week holiday. A round trip across the Atlantic takes most of a decade.

Handy metrics derived from the table

Each rate below comes from one row of the table. Divide the size by the time.

• Hard disk, sequential: 1 MB in 20 ms, so about 50 MB/s. (The older Stanford slide's 30 ms per MB gives 30 MB/s, which is the figure Dean used in his own worked examples.)
• 1 Gbps Ethernet, sequential: 1 MB in 10 ms, so about 100 MB/s. The exact line rate is 125 MB/s; some of it goes to protocol overhead.
• SSD, sequential: 1 MB in 1 ms, so about 1 GB/s.
• Main memory, sequential: 1 MB in 250 µs, so about 4 GB/s.
• Round trips across the world: 1 second ÷ 150 ms ≈ 6 to 7 round trips a second, one after another.
• Round trips within a data center: 1 second ÷ 0.5 ms = 2,000 round trips a second, one after another.

The last two describe one caller waiting for each reply before sending the next request. A server talking to many callers at once can do far more round trips in total.

How these numbers have changed

Hardware has moved on since the table was written, but unevenly. Colin Scott's interactive latency page shows this by letting you pick a year. It starts from Norvig's numbers and applies a trend to each row (source code on GitHub). In its model, CPU clock speed stops rising in 2005, main memory latency stays at 100 ns from 2000 on, the data-center round trip stays at 500 µs, and the California–Netherlands round trip stays at 150 ms because it is limited by the speed of light. Storage and network bandwidth keep growing. Treat the page as a model of trends. Its numbers are extrapolations from older data.

For a few rows that have moved the most, here is a modern ballpark from current vendor specifications and published measurements. The classic table above stays as it is; use these when you want a number closer to today's hardware.

Some notes on reading this table:

• SSDs improved the most. Random reads are about 2 to 3 times faster, and sequential reads about 14 times faster. The 14,500 MB/s figure is a peak for large sequential reads. A data-center SSD also handles millions of small reads a second when many run in parallel; the D7-PS1010 page lists up to 3.3 million random 4 KB reads a second.
• Spinning disks got faster at streaming, but not at seeking. The Seagate figure of 285 MB/s is for the outer edge of the platter, where a disk is fastest. Its 4.16 ms is the average wait for the platter to spin to the right place (half a turn at 7,200 RPM), before counting the head's move. About 170 random reads a second means each one still costs several milliseconds. A disk is still a poor place for random lookups.
• Data-center networks got several times faster. The Google measurement is between two virtual machines in one zone, using the internal network. Your numbers depend on the cloud, the instance type and the load.
• Memory latency barely moved, but bandwidth grew a lot. The 614 GB/s figure is a peak for the whole socket, with all 12 memory channels in use. One thread reading on its own gets only part of it.
• The speed of light did not change. Cross-continent latency is the same as in 2012, which makes it relatively more expensive than everything else that got faster.

Throughput numbers

Throughput is how much work a system finishes per unit of time: bytes per second for a disk, requests per second for a server. The latency table hides throughput inside its rows. This section pulls it out.

Sequential read and transfer rates

A rule of thumb follows from this table. A fast modern SSD reads data about as fast as a 100 Gbps network can carry it, and a hard disk reads about as fast as a 2 Gbps link. So in a data pipeline, the network card often runs out of capacity before the drive does.

Requests per second for one machine

How many requests one machine can handle depends heavily on what each request does. The numbers below are rough orders of magnitude for sizing. Real systems often do much less, so measure your own before you commit to a design.

The app server row is easiest to reason about from first principles. If each request needs about 10 ms of CPU time and the server has 16 cores, the most it can do is 16 × (1,000 ms ÷ 10 ms) = 1,600 requests a second. In practice you plan to run well below that peak, because waiting times rise sharply as a server gets close to fully busy (see Performance and latency). A server that mostly waits on other services, rather than computing, can hold many more requests at once, and then memory or connection limits set the ceiling instead.

The gap between the synthetic database result and a real workload is the main lesson of this table. Benchmarks measure the best case. A design should assume much less, and check against measurements once the system exists.

Numbers for AI systems

AI models add a new kind of estimate: will the model fit in memory, and how fast can it run? The basic numbers come from the GPU's specification and from the model's size.

Memory and bandwidth of one data-center GPU. NVIDIA's spec pages list:

HBM (high-bandwidth memory) is the memory stacked next to the GPU chip. Its bandwidth is several times higher than a whole server CPU socket's memory bandwidth (614 GB/s above).

How big a model's weights are. A model's weights take up roughly:

weight size = number of parameters × bytes per parameter

The bytes per parameter depend on the number format. FP16 and BF16 (16-bit floating point) use 2 bytes. 8-bit formats use 1 byte. 4-bit formats use half a byte. So:

• A 70-billion-parameter model in FP16 takes 70 × 10^9 × 2 bytes = 140 GB.
• The same model in an 8-bit format takes about 70 GB, and in a 4-bit format about 35 GB.

This is only the weights. Running a model also needs memory for intermediate results, mainly the cache of each conversation's past tokens. So a 140 GB model does not fit on one 80 GB H100, and it only just fits within one H200's 141 GB with almost no room left over. In practice, it would run on two or more GPUs.

A rough speed limit. When a model generates text one token at a time for a single user, each new token needs the GPU to read all the weights from memory once. Reading 140 GB at 4.8 TB/s takes about 29 ms. So one H200 can produce at most about 34 tokens a second for that single conversation, however fast its arithmetic units are. Serving many users at once in the same pass over the weights is how real systems get much higher total throughput.

The gist that popularized the latency table added rows for language models in 2026. They are rough, but useful for scale: a hosted frontier model produces a token about every 20 ms, takes about 1 second before its first token on a short prompt, about 10 seconds to read a 100,000-token prompt with no cache, and about 30 seconds for a single reasoning response.

A later part of the handbook covers model inference in depth: batching, caching, and how to size a serving fleet.

Check yourself

Further reading

The classic sources for the latency table:

• Latency numbers every programmer should know: Jonas Bonér's gist, the version most people copy, with credits and later additions.
• "Humanized" latency numbers: a second gist with a human-scale version of the table.
• Visual chart of the latency numbers: the table drawn as grids of squares.
• Teach Yourself Programming in Ten Years: Peter Norvig's essay; the timing table is in its answers section.
• Designs, Lessons and Advice from Building Large Distributed Systems: Jeff Dean's LADIS 2009 keynote. "Numbers Everyone Should Know" and the thumbnail estimate are on slides 24 to 27.
• Software Engineering Advice from Building Large-Scale Distributed Systems: Jeff Dean's earlier Stanford talk, with the older version of the table.
• Latency Numbers Every Programmer Should Know (interactive): Colin Scott's page with a year slider.

Units and modern figures:

• Power of two: Wikipedia's article, with a longer table of powers.
• Prefixes for binary multiples: NIST on KiB, MiB and GiB.
• What a trip! Measuring network latency in the cloud: how to measure round-trip time between machines, and why the settings change the result.
• Redis benchmark: how to benchmark a cache node, and the pitfalls that make benchmarks misleading.
• Back-of-the-envelope estimation: the step-by-step method that uses these numbers.