Network protocols and APIs

This chapter answers five questions:

• What happens between "the app sends a request" and "the server gets it", and what are the layers people refer to?
• What does HTTP promise, and what do its methods and status codes mean?
• When should data travel over TCP, and when over UDP?
• How do RPC and REST differ as ways to design an API, and what does each cost?
• Where do gRPC and GraphQL fit, and how do you keep an API usable as it changes?

We'll keep using Crumb, the recipe site from the Scalability chapter. Crumb now has a website, a mobile app, app servers behind a load balancer, a memcached layer and a database. Every arrow between those boxes is a conversation between two programs on different machines. The rules for such a conversation are called a protocol. The set of requests a server agrees to answer is called its API (application programming interface).

Layers: how one request is built

Think of a letter sent inside several envelopes. You write the letter and put it in an envelope addressed to a person. The mailroom puts that inside a bigger envelope addressed to their building. The postal service adds labels to get it to the right city. Each handler reads only its own envelope and passes the rest along.

Network data is wrapped the same way. When Crumb's app asks for a recipe, the request gets several layers of wrapping on the way out, and the server unwraps them in reverse order. The best-known description of these layers is the OSI model (Open Systems Interconnection), published in 1984 as standard ISO 7498 (Wikipedia). It has seven layers, numbered from the wire upward.

Two warnings make the table more useful.

• The internet uses a simpler model. Its protocols weren't designed to fit OSI. RFC 1122 (1989) describes four layers: link, internet (IP), transport (TCP or UDP) and application. Its application layer "essentially combines the functions of the top two layers" of OSI, and Wikipedia adds most of the session layer too. In practice, people use the OSI numbers as labels and treat 5 to 7 as one layer.
• Some protocols sit between layers. TLS, the encryption behind HTTPS, does presentation-layer work but runs on top of a transport connection. Wikipedia says it "does not strictly fit inside the model."

In system design you'll hear the numbers most often about load balancers. A layer 4 load balancer routes by IP address and port without reading the request. A layer 7 load balancer reads the HTTP request, so it can send /api to one pool of servers and /images to another. Scalability covers both.

HTTP: requests and responses

HTTP (Hypertext Transfer Protocol) is a way to encode data and carry it between a client and a server. It is a request/response protocol. The client sends a request. The server sends back a response with the content asked for and a status code that says how the request went.

Here is Crumb's app asking for a recipe, followed by the server's answer:

GET /recipes/456 HTTP/1.1
Host: api.crumb.example
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: max-age=60

{"id": "456", "title": "Lemon drizzle cake", "authorId": "1234"}

The first line holds the two essentials: a method, also called a verb (GET), and a resource, also called an endpoint (/recipes/456). The lines after it are headers, which carry extra information such as the format the client wants. A request can also carry a body, as a POST usually does.

Every message stands on its own

HTTP is stateless. RFC 9110, the current definition of HTTP, says "each request message's semantics can be understood in isolation." The server doesn't need to remember earlier requests to understand this one. If Crumb needs to know who is logged in, each request carries that itself, usually in a cookie or a token.

Because each message is self-contained, other machines can sit between client and server and act on it. RFC 9110 calls them intermediaries. A load balancer can send each request to a different server. A cache can answer a repeated GET itself. A proxy can encrypt or compress traffic. The RFC notes that "many implementations depend on HTTP's stateless design in order to reuse proxied connections or dynamically load balance requests across multiple servers." That is why the stateless app servers from the Scalability chapter fit HTTP so well.

Methods and their properties

Three properties, defined in RFC 9110 section 9.2, decide how clients, caches and proxies may treat a method:

• Safe means read-only. The client isn't asking for anything to change, so a search engine crawler can follow safe links without harm.
• Idempotent means sending the same request many times has the same intended effect as sending it once. This is what makes a request safe to retry after a timeout.
• Cacheable means a cache may store the response and reuse it for later requests.

These values follow RFC 9110 and RFC 5789. Four details are worth knowing:

1. Cacheable POST and PATCH responses are rare. RFC 9110 lets a cache store a POST response only if it has explicit freshness information and a Content-Location header equal to the URL that was posted to. Even then, the copy can answer a later GET or HEAD, never another POST. RFC 5789 sets the same rule for PATCH. RFC 9110 also notes that "the overwhelming majority of cache implementations only support GET and HEAD." In practice, treat only GET as cacheable.
2. Idempotent doesn't mean the same response. Deleting recipe 456 twice leaves the same state as deleting it once. The first call might return 204 No Content and the second 404 Not Found. RFC 9110 says the effect is the same "though the response might differ."
3. PATCH can be idempotent. RFC 5789 says PATCH "is neither safe nor idempotent" in general, because "add 1 to the like count" changes the result each time. "Set the title to X" is idempotent. A client can also make a PATCH conditional, so it fails if the resource changed since the client last read it.
4. PUT and DELETE clear caches. Their responses are never cached, and when one succeeds, caches it passes through mark their stored copy of that URL as out of date.

Two more methods are worth knowing. HEAD is a GET that returns only headers, and OPTIONS asks which methods a resource supports. Both are safe.

Idempotence matters because RFC 9110 says a client "SHOULD NOT automatically retry a request with a non-idempotent method" unless it knows the retry is harmless. If Crumb's app sends POST /recipes and the connection drops before the reply, the app can't tell whether the recipe was saved. A retry might create a duplicate. Idempotency keys, later in this chapter, fix that. How caches decide what to store and for how long, using Cache-Control and ETag, is covered in the DNS and CDNs chapter.

Status codes

Every response has a three-digit status code. RFC 9110 says "the first digit of the status code defines the class of response":

Two pairs are often confused. 401 means the request "lacks valid authentication credentials", so the client should log in. 403 means the server understood the request and "refuses to fulfill it." 429 comes from RFC 6585 and means the client is being rate limited. Rate limiting covers how to tell clients that.

The class tells a client whether to retry. A 400 will fail the same way every time. A 503 might succeed later.

HTTP needs a transport underneath

HTTP is an application-layer protocol. It defines what messages mean and leaves moving the bytes to a lower-level transport protocol. For most of its history that has been TCP. Since HTTP/3 it can also be QUIC, which runs on UDP. The next sections cover those transports and then how the HTTP versions use them.

TCP: reliable delivery, with a price

TCP (Transmission Control Protocol) is like registered mail with tracking. Both sides agree to talk before anything is sent. Every piece is numbered, the receiver signs for what arrives, and anything lost is sent again. The receiving program gets all the data in the right order, or learns that the connection failed.

TCP is connection-oriented and runs over IP (Internet Protocol), which moves single packets between machines with no delivery promise. RFC 9293, the current TCP standard, sums up what TCP adds: "a reliable, in-order, byte-stream service to applications."

The three-way handshake

Before any data flows, the two sides open a connection with a three-way handshake. These numbers come from RFC 9293's own example:

1. SYN. The client sends a segment with the SYN ("synchronize") flag set, saying "I'll number my bytes starting at 100."
2. SYN-ACK. The server replies with its own SYN, "I'll start at 300", and an ACK ("acknowledge") of 101, meaning "I got your 100 and expect 101 next."
3. ACK. The client acknowledges with 301. The connection is open, and the client's next segment can carry data.

The RFC says the main purpose is to stop "old duplicate connection initiations from causing confusion": a stray SYN from an earlier attempt can't open a ghost connection. To close, each side sends a segment with the FIN flag ("no more data from sender") and the other side acknowledges it.

The handshake costs a full round trip before the request can be sent. If a Crumb user is 150 ms away, that's 150 ms before the app has asked for anything. Performance and latency explains why round trips, more than bandwidth, decide how fast small requests are.

How TCP keeps data intact and in order

RFC 9293 says TCP detects "packet losses (via sequence numbers) and errors (via per-segment checksums)" and corrects them "via retransmission":

• Sequence numbers let the receiver put segments back in order and spot gaps.
• Checksums, computed from each segment's contents, reveal bits flipped in transit. A damaged segment is dropped.
• Acknowledgements tell the sender which bytes arrived.
• Retransmission resends data that wasn't acknowledged in time.

TCP doesn't retry forever. After repeated timeouts on the same data, it closes the connection. RFC 9293 says that limit should be at least 100 seconds for data.

Flow control and congestion control

• Flow control protects the receiver. Each segment carries a window, the number of bytes its sender "is willing to accept." A slow receiver shrinks its window, and the other side slows to match.
• Congestion control protects the network. RFC 9293 requires "slow start, congestion avoidance, and exponential backoff" (detailed in RFC 5681). A new connection sends a little and speeds up as acknowledgements return. When packets are lost, TCP takes that as a sign of overload and slows down.

Together these let TCP find a good sending rate on its own, without knowing the network's capacity in advance.

What the guarantees cost

• Setup time. The handshake adds a round trip, and TLS adds more (see below).
• Waiting for lost data. TCP delivers bytes in order. If segment 5 is lost, segments 6 to 20 wait until segment 5 is resent, even if they arrived. One missing piece holds up everything behind it. This is called head-of-line blocking.
• Slower sending after a loss, because congestion control backs off.
• Memory per connection on both machines, for buffers and state.

So TCP generally moves data less efficiently than UDP. And because web servers keep many TCP connections open to get high throughput, the memory adds up.

Facebook hit this limit with its cache. In Scaling Memcache at Facebook (NSDI 2013), its engineers wrote that "the high memory demands of open TCP connections makes it prohibitively expensive to have an open connection between every web thread and memcached server." So each web server sends cache writes through a proxy called mcrouter on the same machine. Mcrouter coalesces the connections, so many threads share a few connections to each cache server.

This is connection pooling: open a set of connections once and lend them to requests as needed. It saves a handshake per request and caps memory. Crumb does the same for its database, and Little's law tells you how big the pool should be. Facebook also took a second route for cache reads: it moved them to UDP.

When to use TCP

TCP suits programs that need high reliability and can accept some delay: web servers, database connections, email (SMTP), file transfer (FTP) and remote login (SSH). Use TCP when:

• you need all of the data to arrive intact, and
• you want the protocol to make the best use of the network's capacity for you.

UDP: fast, with no promises

UDP (User Datagram Protocol) is like sending postcards. You address each card and drop it in the box. There's no setup and no tracking. Most cards arrive, some arrive out of order, some never arrive, and nobody tells you.

RFC 768 defined UDP in 1980, in three pages. It sends messages "with a minimum of protocol mechanism," and "delivery and duplicate protection are not guaranteed." The header is 8 bytes: source port, destination port, length and checksum. A sender may even skip the checksum.

A UDP message is called a datagram, the UDP word for a packet. Each one is handled on its own, and the only promise is at the datagram level: a datagram that arrives is whole. Datagrams may arrive out of order or not at all. UDP has no connection, no retransmission and no congestion control. Without TCP's guarantees there's less work and less waiting, so UDP is generally more efficient.

Broadcast, and why DHCP needs it

UDP can broadcast: send one datagram to every device on the local network. TCP can't, because a TCP connection has exactly two ends.

This solves a chicken-and-egg problem. A laptop joining Wi-Fi uses DHCP (Dynamic Host Configuration Protocol) to get an IP address. Until it has one, it can't open a TCP connection to anybody. RFC 2131 says DHCP runs on UDP. The client, with its own address set to zero, broadcasts a DHCPDISCOVER message to the all-ones broadcast address on UDP port 67. Any DHCP server that hears it can reply with an offer.

Where UDP wins

UDP is less reliable, but it suits real-time uses where fresh data matters more than complete data: voice calls (VoIP), video chat, live streaming and multiplayer games.

Glenn Fiedler explains why in UDP vs. TCP (2008). A game server sends every player's position many times a second. Over TCP, one lost update holds back the newer ones, which wait in a queue where "you cannot access it until that lost packet has been retransmitted." The game freezes, then jumps. Over UDP, the game uses the newest datagram and moves on. Fiedler advises building only the reliability you need, such as resending important events, on top of UDP yourself.

Facebook's cache reads are a server-side example. The NSDI paper says "we rely on UDP for get requests to reduce latency and overhead," which cut latency by about 20%. At peak, 0.25% of reads were dropped or arrived out of order, and clients treat those as cache misses. Writes and deletes still go over TCP, because losing them isn't acceptable.

When to use UDP

Use UDP when:

• you need the lowest latency,
• late data is worse than lost data, and
• you're willing to build your own error handling where you need it.

TLS and the three versions of HTTP

TLS on top of TCP

Plain TCP carries bytes anyone along the path can read. TLS (Transport Layer Security) encrypts them and proves the server's identity. HTTPS is HTTP inside TLS inside TCP. TLS runs its own handshake after TCP's. In TLS 1.3 (RFC 8446, 2018) that takes one round trip, and a client returning to a recent server can send data immediately in a "0-RTT" mode, "at the cost of certain security properties." So a new HTTPS connection over TCP spends two round trips before the request goes out. Security covers what TLS protects.

HTTP/1.1: keep the connection open

HTTP/1.1, first published in January 1997 (MDN), made persistent connections (keep-alive) the default, "allowing multiple requests and responses to be carried over a single connection." That saves a handshake per request.

A persistent connection still handles one exchange at a time in practice. HTTP/1.1 allows pipelining, sending several requests without waiting, but the server "MUST send the corresponding responses in the same order." One slow response holds up all the others: head-of-line blocking at the HTTP layer. So clients open several connections to the same server, and RFC 9112 warns that "each connection consumes server resources."

HTTP/2: many streams on one connection

HTTP/2, standardized in 2015 and revised as RFC 9113 in 2022, splits messages into binary frames and gives each request its own stream. Frames from many streams are interleaved on one TCP connection. Crumb's recipe page can fetch the recipe, photos and comments at once, and a slow response no longer blocks the others. HTTP/2 also encodes headers compactly.

RFC 9113 states the catch: "TCP head-of-line blocking is not addressed by this protocol." All the streams share one TCP byte stream. When one packet is lost, TCP holds back everything after it, including frames for streams that lost nothing. As the HTTP/3 standard puts it, "a lost or reordered packet causes all active transactions to experience a stall."

HTTP/3: streams move into the transport

HTTP/3 (2022) keeps HTTP's meaning and runs it over QUIC (RFC 9000, 2021). QUIC is a reliable transport that knows about streams, providing "reliability at the stream level and congestion control across the entire connection." In MDN's words, "only the stream with data in that packet is blocked." QUIC also:

• builds in TLS 1.3, so the transport and encryption handshakes happen together instead of one after the other, and
• identifies a connection by an ID instead of an IP address and port, so a phone can move from Wi-Fi to mobile data without starting over.

Why build reliability on top of unreliable UDP? RFC 9000 says QUIC uses UDP "to better facilitate deployment in existing systems and networks." Routers, firewalls and operating systems already pass UDP. A new transport protocol would be blocked by many of them. A server advertises HTTP/3 with an Alt-Svc response header, and the client switches on later requests.

All three versions share the same methods, status codes and headers. When Crumb's load balancer starts speaking HTTP/3 to phones, the API code doesn't change.

RPC: calling a function on another machine

In a remote procedure call (RPC), a client makes a procedure run in a different address space, usually on a remote server. The call is written as if it were local, and the details of reaching the server are hidden from the client program. On Crumb, an API server might get nutrition facts from a separate nutrition service like this:

facts = nutrition.get_facts(recipe_id="456")

It reads like a local function call, but it behaves differently. A local call takes nanoseconds and never half-fails. A remote call takes milliseconds, can time out, and can fail after the server already did the work. Because remote calls are slower and less reliable, it helps to make them easy to tell apart from local calls, for example by naming and by giving every call an explicit timeout.

How a call travels

Generated code called stubs makes the remote call look local. A call goes through these steps:

1. Client program. It calls the client stub. The arguments are pushed onto the stack, just as for a local call.
2. Client stub. It marshals (packs) the procedure's identifier and the arguments into a request message.
3. Client communication module. The operating system sends the message to the server.
4. Server communication module. The server's operating system passes the incoming packets to the server stub.
5. Server stub. It unmarshals (unpacks) the message and calls the procedure matching the identifier, passing the arguments.
6. The reply repeats these steps in reverse, back to the client program.

Tyson Trautmann's Debunking the Myths of RPC & REST (2012) adds that "nothing about RPC is inherently bound to HTTP." RPC can use HTTP to carry messages, or its own protocol.

Frameworks generate the stubs from a description of the service. Three popular ones:

• Protocol Buffers (Protobuf), from Google, is "a language-neutral, platform-neutral extensible mechanism for serializing structured data," "like JSON, except it's smaller and faster." gRPC uses it.
• Apache Thrift pairs a data format with "a code generation engine" for many languages. Facebook built it in 2006 and open-sourced it in 2007.
• Apache Avro is "a data serialization system" that also supports RPC. Data files carry their schema, so "code generation is not required."

What RPC calls look like over HTTP

The URL names an action:

GET /getRecipe?recipeId=456

POST /rateRecipe
Content-Type: application/json

{"recipeId": "456", "userId": "1234", "stars": 5}

RPC is focused on exposing behaviors. It is often used for internal communication for performance reasons, because you can hand-craft calls to fit each use case and use a compact binary format.

The same reasoning applies to offering a native library (an SDK) that customers call from their own code. Choose one when:

• you know the platform your users are on,
• you want to control how your logic is accessed,
• you want to control how errors are handled on the library's side, and
• performance and the end user's experience are your main concern.

For public APIs called by unknown developers in any language, HTTP APIs in the REST style are more common.

gRPC: RPC over HTTP/2

Google long used an internal RPC system called Stubby. In March 2015 it decided to build the next version as open source, called gRPC. It lets a client "call a method on a server application on a different machine as if it were a local object" (introduction). You describe the service in a .proto file:

syntax = "proto3";

service NutritionService {
  rpc GetFacts (GetFactsRequest) returns (NutritionFacts);
  rpc WatchPrices (WatchPricesRequest) returns (stream IngredientPrice);
}

message GetFactsRequest {
  string recipe_id = 1;
}

A compiler generates client and server code in each language. On the wire, gRPC runs over HTTP/2: every call is a POST to a path like /NutritionService/GetFacts, with content type application/grpc and a body of length-prefixed Protobuf messages. The result code arrives in trailers, headers sent after the body.

HTTP/2 streams give gRPC four kinds of method:

gRPC also builds in two habits remote calls need. Deadlines let the client say how long it will wait, after which the call fails with DEADLINE_EXCEEDED. Cancellation lets either side stop a call, though "changes made before a cancellation are not rolled back."

Browsers can't call gRPC directly, because "raw HTTP/2 frames are inaccessible in browsers" (gRPC blog). The workaround, gRPC-Web, needs a translating proxy. So most teams use gRPC between their own services and JSON over HTTP for browsers, often with an API gateway translating between them (Reverse proxies and API gateways).

Disadvantages of RPC

• Clients become tightly coupled to the service. Clients call specific procedures with specific arguments. If the nutrition team renames GetFacts or changes its arguments, clients must be rebuilt and redeployed in step.
• Every new operation needs a new API. Facts for ten recipes at once means a new GetFactsBatch. Trautmann warns that hand-rolled interfaces tend to be "too narrow for future use cases," leading to "an explosion in the number of methods."
• RPC can be hard to debug. Binary messages can't be read in a log or replayed with curl.
• Existing tools may not work out of the box. An HTTP cache such as Squid understands GET and URLs. Trautmann notes that RPC inputs may not be passed "in a standard way that makes it possible to map requests to cached responses," so caching RPC results takes extra work. gRPC sends every call as a POST, so standard HTTP caches won't store any of them.

REST: resources and representations

REST (Representational State Transfer) is an architectural style, meaning a set of design rules a system can follow. Roy Fielding defined it in chapter 5 of his 2000 dissertation.

REST uses a client/server model. The client acts on a set of resources that the server manages. Fielding says "any information that can be named can be a resource": a recipe, a user, a user's saved recipes. The server provides representations of resources, such as a JSON document describing a recipe, and actions that either change resources or return new representations.

All communication must be stateless and cacheable. Each request "must contain all of the information necessary to understand the request," and responses must be labeled as cacheable or not, so clients and intermediaries know when they can reuse one.

The four qualities of a RESTful interface

Fielding's central rule is a uniform interface: every resource is handled the same way. It has four parts:

Most APIs that call themselves REST skip the last one. Fielding complained in 2008 about "the number of people calling any HTTP-based interface a REST API" (REST APIs must be hypertext-driven). His rule is that "a REST API should be entered with no prior knowledge beyond the initial URI." A client that follows links instead of building URLs keeps working when the server reorganizes them.

What REST calls look like

GET /recipes/456

HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": "456",
  "title": "Lemon drizzle cake",
  "links": {"author": "/users/1234", "comments": "/recipes/456/comments"}
}

PUT /recipes/456
Content-Type: application/json

{"title": "Lemon drizzle cake (gluten-free)", "servings": 8}

The URLs are nouns and the method carries the action. REST is focused on exposing data.

Why REST suits public APIs

• It minimizes coupling. A client needs only the URIs and the standard methods. Crumb can change how it stores recipes without any client noticing, so REST is the usual choice for public HTTP APIs.
• It is uniform. Resources are exposed through URIs, representations are described in headers, and actions use the verbs GET, POST, PUT, DELETE and PATCH. Browsers, curl, caches, CDNs and proxies already understand all of it.
• It scales. Because requests are stateless, any server can handle any request. That makes REST services easy to scale horizontally behind a load balancer and to partition across servers. Fielding lists scalability among the benefits of statelessness.

Disadvantages of REST

• Not every question fits a resource path. REST works best when data forms a simple hierarchy, like /users/1234/recipes. "Every recipe updated in the past hour that matches these dietary tags" doesn't. It ends up as a mix of path, query parameters and maybe a request body: GET /recipes?updatedSince=2026-10-17T09:00Z&tags=vegan.
• A few verbs may not fit the action. "Move all expired recipes into the archive" isn't clearly a create, replace or delete. Teams fall back on workarounds like POST /recipes/archive-jobs.
• Nested data takes many round trips. One recipe page might need the recipe, then its author, then its comments: several requests, often one after another. On a phone with a weak signal, each round trip hurts.
• Responses grow. As fields are added over time, old clients receive all of them, including ones they never use. Payloads get bigger and slower.

Fielding named the trade-off himself: "a uniform interface degrades efficiency, since information is transferred in a standardized form rather than one which is specific to an application's needs."

RPC and REST side by side

The same operations on Crumb, designed both ways. User 1234 saves recipe 456 to their list.

RPC names every action, so the endpoint list grows with each feature. REST keeps a small set of resource URLs and lets the method do the work. REST's methods also tell intermediaries something: a cache knows it can store the GET, and a retry library knows the PUT and DELETE are safe to repeat. Every RPC write is a POST, which tells them nothing.

Arnaud Lauret's Do you really know why you prefer REST over RPC? (2015) argues that neither wins in general. RPC can be easier when you mostly expose operations, and REST when you "deal mainly with data." One API can mix both.

GraphQL: the client picks the fields

Two of REST's problems have names. Over-fetching means getting fields you don't need. Under-fetching means getting too little per response, so you need more round trips. GraphQL addresses both. Its introduction defines it as "a query language for your API, and a server-side runtime for executing queries using a type system you define for your data."

A GraphQL server usually has a single endpoint, typically /graphql. The client sends a query shaped like the data it wants, so Crumb's recipe page becomes one request:

{
  recipe(id: "456") {
    title
    author { name }
    comments(first: 10) { text author { name } }
  }
}

The client gets "just that data in the expected shape with a single request." A phone can ask for fewer fields than the website, and old clients never receive fields added later. Instead of versions, you add fields and mark old ones @deprecated until nobody uses them.

The costs:

• HTTP caching is harder. The caching guide notes "there's no URL-like primitive" identifying each object. Writes must use POST, and long queries can exceed URL limits for GET.
• Status codes say less. A response with some data and some errors still returns 2xx, because "HTTP does not yet have a status code representing 'partial success.'"
• One query can fan out to many tables and services, so the server must resolve each field efficiently.

If you only need a tailored response for one kind of client, a Backend for Frontend (BFF) is an alternative: a small server per client type that calls your APIs and returns what each screen needs (Reverse proxies and API gateways).

Designing an API that lasts

Versioning

Once an app is on millions of phones, you can't update every client at once, so you need to know which changes break them. GitHub's versioning policy is a clear model. "Adding an optional parameter" is additive and appears in every supported version. "Removing or renaming a parameter" or "changing the type of a parameter or response field" is breaking, so it goes into a new version.

Three common ways to choose a version:

• In the URL, such as /v2/recipes/456. Simple and visible, but each version is a new set of URLs.
• In a header. GitHub uses X-GitHub-Api-Version with date-based names and supports old versions "for at least 24 more months." Stripe uses Stripe-Version, releasing monthly versions without breaking changes and a breaking major version twice a year.
• Never break, only add. GraphQL takes this approach, and Protobuf supports it: old code "will read new messages without issues, ignoring any newly added fields" (overview).

Pagination

"List recipe 456's comments" can't return a million rows. APIs return a page plus a way to get the next. Offsets (?offset=40&limit=20) are simple but get slower deep in a list and can skip or repeat items as new ones arrive. A cursor, an opaque token pointing just past the last item seen, avoids both. Designing a social news feed explains in detail.

Idempotency keys make POST safe to retry

To make a POST safe to retry, the client generates a unique idempotency key per operation and sends it with every attempt, usually in an Idempotency-Key header. The server stores the result under that key and, on a repeat, returns the stored result instead of doing the work again. Stripe's rules are a good template:

• Keys should be V4 UUIDs or similarly random strings, up to 255 characters.
• The first result is saved and replayed on retries, "including 500 errors."
• A retry whose parameters differ from the original's gets an error, which catches reused keys.
• Keys may be removed once they are at least 24 hours old.
• GET and DELETE don't need keys, because they "are idempotent by definition."

An IETF draft to standardize the Idempotency-Key header expired without becoming an RFC, so it remains a convention. Reliability patterns shows how to store and replay results.

Choosing between REST, gRPC and GraphQL

Crumb uses each where it fits. The public API and website use REST over HTTPS, so the CDN can cache GET /recipes/456. Internal services call each other with gRPC, with deadlines on every call. The mobile app could move to GraphQL or a BFF if nested REST calls become a real cost. Live features, like a new comment appearing as you read, need a long-lived connection, which Realtime covers.

Check yourself

Further reading

Specifications:

• RFC 9110: HTTP Semantics: the definitive source for methods and status codes. Section 9 covers methods.
• RFC 9293: TCP: the 2022 TCP standard. Section 3.5 has the handshake example used above.
• RFC 768: UDP: all of UDP in three pages.
• RFC 9113 (HTTP/2) and RFC 9114 (HTTP/3): their introductions explain in a few paragraphs why each version exists.
• RFC 5789: PATCH: short, and clear on why PATCH is neither safe nor idempotent.

HTTP, TCP and UDP:

• Evolution of HTTP: MDN's history from HTTP/0.9 to HTTP/3.
• What is HTTP?: a short glossary entry from F5. The old NGINX link redirects here.
• What's the difference between PUT and PATCH?: a forum thread with practical examples of full and partial updates.
• UDP vs. TCP: Glenn Fiedler on why real-time games avoid TCP.
• Difference between TCP and UDP: a Stack Overflow thread with short, clear answers.
• Transmission Control Protocol and User Datagram Protocol: Wikipedia's articles, with header diagrams and connection states.
• Scaling Memcache at Facebook: the NSDI 2013 paper. Section 3.1 covers UDP reads, TCP writes and shared connections.

RPC and REST:

• Fielding's dissertation, chapter 5: the original definition of REST, readable without the rest of the dissertation.
• REST APIs must be hypertext-driven: Fielding on what most "REST" APIs leave out.
• Do you really know why you prefer REST over RPC?: Arnaud Lauret's operation-by-operation comparison.
• Debunking the Myths of RPC & REST: Tyson Trautmann on why HTTP caching works for REST and not for RPC (archived copy).
• REST vs JSON-RPC?: a Stack Overflow discussion of the trade-offs.
• Why REST for internal use and not RPC: an Ars Technica forum thread on the same question for internal services (archived copy).
• HATEOAS: the REST Cookbook's short page on hypermedia links (archived copy).
• Know your HTTP headers: a table of HTTP headers with links to the specs that define them.
• Crack the system design interview: an interview walkthrough of a photo-sharing service, including how services find each other. The old puncsky.com link redirects here.
• Building and open-sourcing fbthrift: Facebook's account of where Thrift came from.

gRPC, GraphQL and API design:

• gRPC core concepts and gRPC over HTTP/2: the method kinds, deadlines, and how calls map onto HTTP/2.
• The state of gRPC in the browser: why browsers need gRPC-Web and a proxy.
• Introduction to GraphQL and GraphQL caching: the official guide, including where caching gets harder.
• Stripe idempotent requests, Stripe versioning and GitHub API versions: real public APIs' rules, worth copying.