Idempotency Keys, Retries, Webhooks and Rate Limits

Think about the last time you tapped "Pay" and the screen froze. Did the payment go through? You did not know. So you tapped again. Now you may have paid twice.

That is the whole story of this lesson. The request left your phone. The server may have done the work. The answer never came back. From the client side, "it failed" and "it worked but I did not hear" look exactly the same. Good APIs are built so the client can try again safely.

This is lesson 5 of the API design series. In lesson 0 we named resources, in lesson 1 we saw which HTTP methods are idempotent (GET, PUT, DELETE) and which are not (POST), and in lesson 4 we changed an API without breaking clients. Here we fix the hardest gap: POST is not idempotent, but clients must retry it anyway. Next, lesson 6 covers who is allowed to call all this: authentication and authorization.

Words used in this lesson

• Retry means sending the same request again because you did not get a clear answer.
• Idempotent means doing it many times has the same effect as doing it once. PUT /users/7 with the same body is idempotent. POST /payments is not, unless we add a key.
• Idempotency key is a unique string the client puts in a header (Idempotency-Key) to say "this is the same attempt as before".
• Fingerprint is a short hash of the request, used to check that a repeated key really comes with the same request.
• Backoff means waiting longer after each failed try (1s, 2s, 4s...). Jitter means adding randomness to the wait, so many clients do not all retry at the same moment.
• Webhook is a request your server sends to the customer's server when something happens. It is an API call in the other direction.
• HMAC is a way to sign a message with a shared secret. Whoever knows the secret can check that the message was not changed.
• Replay attack means someone records a real message and sends it again later.
• Rate limit is a cap on how many requests a client may send in a time window.
• 429 is the status code "Too Many Requests". Retry-After is a header that says how long to wait.
• 202 Accepted means "I got your request and will do it later".
• Token bucket is a rate-limit method: a bucket fills with tokens at a steady speed, each request takes one token.

The problem: the lost answer

Here is a tiny payments API. A client posts a payment, the server talks to a slow bank (we fake it with a 100 ms wait), and replies 201. The client has a timeout of 50 ms. So the client gives up early and, being a good citizen, tries once more.

// idem1.mjs  (shortened: the full server is 15 lines)const charges = [];const naive = http.createServer(async (req, res) => {  if (req.method === "POST" && req.url === "/payments") {    let body = ""; for await (const c of req) body += c;    const { amount } = JSON.parse(body);    await new Promise(r => setTimeout(r, 100));          // slow bank    const charge = { id: "ch_" + (charges.length + 1), amount };    charges.push(charge);    res.writeHead(201, { "content-type": "application/json" });    return res.end(JSON.stringify(charge));  }  res.writeHead(404).end();});

The client sends the request with AbortSignal.timeout(50), gets a timeout error, and posts again. Real output:

first try: TimeoutErrorretry status: 201charges stored: [ { id: 'ch_1', amount: 500 }, { id: 'ch_2', amount: 500 } ]

The customer paid 500 twice. Nobody did anything wrong. The client behaved sensibly, the server behaved sensibly, and still we have two charges. The bug is in the contract: the API has no way to say "this is the same payment as before".

Which methods are in danger? From lesson 2: GET, PUT and DELETE are idempotent by definition, so a retry is safe if you built them correctly. POST (create something, run an action) and often PATCH are not. These are exactly the calls that move money, send emails and create orders.

The idea: an idempotency key

The fix is simple to say. The client makes up a random unique string before the first try and sends it in a header:

POST /payments HTTP/1.1Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324Content-Type: application/json{"amount": 500}

If the connection breaks, the client sends the same request with the same key. The server looks at the key:

• Never seen it: do the work, save the result under the key, reply.
• Seen it, work finished: do not do the work again. Reply with the saved result.
• Seen it, work still running: tell the client "wait", with 409 Conflict.
• Seen it, but the request body is different: this is a client bug. Reject it with 422.

This is what the IETF draft for the Idempotency-Key header describes. I read it. Key points from the draft, in my words:

1. The header's value is a string. The draft recommends a UUID or another random value.
2. The key must not be reused with a different request body.
3. The server decides how long keys live (the draft says to publish the expiry policy in your docs).
4. The server may use a fingerprint, a hash of the request, to check that "same key" means "same request".
5. For a missing key on an operation that needs one, reply 400. For a reused key with a different body, reply 422. For a retry while the first request is still running, reply 409. The draft shows the error bodies as problem details (the format from lesson 2).
6. Without prior knowledge, a general client cannot assume the server respects the header. So you must document which endpoints use it.

Now the real server. Read the comments, they carry the important ideas.

// idem2.mjsimport http from "node:http";import { createHash } from "node:crypto";const charges = [];const keys = new Map(); // key -> { fp, state: "running"|"done", status, body }const fingerprint = (method, path, body) =>  createHash("sha256").update(method + " " + path + " " + body).digest("hex");const send = (res, status, obj, extra = {}) => {  res.writeHead(status, { "content-type": "application/json", ...extra });  res.end(JSON.stringify(obj));};const server = http.createServer(async (req, res) => {  if (!(req.method === "POST" && req.url === "/payments")) return send(res, 404, { error: "not found" });  let raw = ""; for await (const c of req) raw += c;  const key = req.headers["idempotency-key"];  if (!key) return send(res, 400, { error: "Idempotency-Key header is required" });  const fp = fingerprint(req.method, req.url, raw);  const seen = keys.get(key);  if (seen) {    if (seen.fp !== fp) return send(res, 422, { error: "key reused with a different request" });    if (seen.state === "running")      return send(res, 409, { error: "request with this key is still running" }, { "retry-after": "1" });    return send(res, seen.status, seen.body, { "idempotent-replayed": "true" });  }  keys.set(key, { fp, state: "running" }); // claim the key BEFORE doing the work  try {    const { amount } = JSON.parse(raw);    await new Promise(r => setTimeout(r, 100));      // slow bank    const charge = { id: "ch_" + (charges.length + 1), amount };    charges.push(charge);    keys.set(key, { fp, state: "done", status: 201, body: charge });    send(res, 201, charge);  } catch (e) {    keys.delete(key); // failed before any effect: let the client try again    send(res, 400, { error: "bad json" });  }});

I ran a client against it. It sends the first request with key k-1 and a 50 ms timeout, then retries at several moments, then tries two parallel posts with a new key k-2:

first try with key k-1: TimeoutErrorretry while first still running    409  {"error":"request with this key is still running"}retry after first finished         201 true {"id":"ch_1","amount":500}same key, different amount         422  {"error":"key reused with a different request"}no key at all                      400  {"error":"Idempotency-Key header is required"}two parallel posts, same new key:  parallel B                       409  {"error":"request with this key is still running"}  parallel A                       201  {"id":"ch_2","amount":700}charges stored: [ { id: 'ch_1', amount: 500 }, { id: 'ch_2', amount: 700 } ]

Read it line by line:

• The retry that arrived while the first was still working got 409. It did not start a second charge.
• The retry after the first finished got the same charge (ch_1) back, marked with a header idempotent-replayed: true (the true in the output). The client sees a normal 201, so its code needs no special case.
• Reusing the key k-1 with amount 999 got 422. This catches a real bug: a developer who reuses one key for everything would otherwise silently get the old answer for a new request.
• Two requests at the same instant with the same new key: one won, one got 409. In the end there are exactly two charges for the two different payments (k-1 and k-2), not four.

The race that breaks naive versions

The most common bug is to check the key, then do something slow, then save the key. In the gap another request sneaks through. Here is the buggy version and its output:

// idem3.mjs  (the bug)const server = http.createServer(async (req, res) => {  const key = req.headers["idempotency-key"];  if (done.has(key)) { res.writeHead(200); return res.end(JSON.stringify(done.get(key))); }  await lookup();                      // <-- gap: another request can pass the check above  const charge = { id: "ch_" + (charges.length + 1) };  charges.push(charge); done.set(key, charge);  res.writeHead(201); res.end(JSON.stringify(charge));});

Three parallel requests with the same key:

[ 201, 201, 201 ] charges: 3

Three charges, even though we "have idempotency". The fix is in the good version above: claim the key in the same step as you check it, before any await. In Node, code between two awaits runs without interruption, so keys.get followed directly by keys.set is safe inside one process.

Design decisions for your own API

Client side: retry the right way

Servers do their part; clients must too. A naive retry loop makes outages worse, because 10,000 clients all retry at the same second ("retry storm"). A good retry follows four rules:

1. Retry only what is safe to retry. GET, PUT, DELETE: yes. POST: only if you send an idempotency key.
2. Retry only temporary errors: timeouts, connection resets, 429, 502, 503, 504, and often 500. Never retry 400, 401, 403, 404, 422. They will fail the same way again.
3. Wait with exponential backoff and jitter. Double the maximum wait each time, and pick a random point inside it. The random part is "full jitter".
4. Obey Retry-After. If the server says wait 1 second, do not retry in 100 ms.

// retry.mjsconst sleep = (ms) => new Promise(r => setTimeout(r, ms));const RETRYABLE = new Set([429, 500, 502, 503, 504]);async function fetchWithRetry(url, init, { retries = 5, baseMs = 100, capMs = 2000 } = {}) {  const t0 = Date.now();  for (let attempt = 0; ; attempt++) {    const r = await fetch(url, init);    if (!RETRYABLE.has(r.status) || attempt === retries) {      console.log(`final ${r.status} after ${attempt + 1} attempts, ${Date.now() - t0}ms`);      return r;    }    const ra = r.headers.get("retry-after");    const full = Math.random() * Math.min(capMs, baseMs * 2 ** attempt);   // full jitter    const wait = ra !== null ? Math.max(Number(ra) * 1000, full) : full;    // never sooner than asked    console.log(`attempt ${attempt + 1}: ${r.status}, Retry-After=${ra ?? "-"}, waiting ${Math.round(wait)}ms`);    await sleep(wait);  }}

I pointed it at a server that answers 503 (with Retry-After: 1), then 500, then 429, then 200:

attempt 1: 503, Retry-After=1, waiting 1000msattempt 2: 500, Retry-After=-, waiting 168msattempt 3: 429, Retry-After=0, waiting 257msfinal 200 after 4 attempts, 1462ms

Notice the first wait was exactly 1000 ms because the server asked for it. The next two waits are random (168 and 257 ms); run it again and you will see different numbers. That is the jitter at work.

Retries on the server side need care too. If your API calls another API, and your caller retries you, and you retry them, the retries multiply (3 x 3 x 3 = 27 calls). Retry at one layer, usually the edge, and pass the same idempotency key downstream.

Long-running work: 202 Accepted and a job resource

Some work takes longer than a request should wait: building a report, resizing a video, importing a file. Holding the connection for 90 seconds is a bad idea. Proxies close idle connections, mobile networks drop, and the client times out and retries (hello, double work).

The pattern is called asynchronous request-reply:

1. The client posts the work.
2. The server stores a job, replies at once with 202 Accepted, and puts the job's URL in the Location header.
3. The client polls the job URL (or gets a webhook). The job says running, then done with a link to the result, or failed with a problem details body.

// job.mjs (the handler)if (req.method === "POST" && req.url === "/reports") {  const id = "job_" + (++n); jobs.set(id, { status: "running" });  setTimeout(() => jobs.set(id, { status: "done", result: "/downloads/" + id + ".csv" }), 300);  return send(res, 202, { id, status: "running" }, { location: "/jobs/" + id, "retry-after": "1" });}

Real output from the client that posts, then polls every 150 ms:

202 /jobs/job_1 { id: 'job_1', status: 'running' }poll 1 200 {"id":"job_1","status":"running"}poll 2 200 {"id":"job_1","status":"running"}poll 3 200 {"id":"job_1","status":"done","result":"/downloads/job_1.csv"}

A few rules that make this pattern pleasant:

• The job is a resource. GET /jobs/job_1 is a normal, cacheable-for-a-second, idempotent read.
• Say how often to poll with Retry-After, so clients do not hammer you.
• Give the job states a fixed list: queued, running, done, failed, maybe cancelled. Support DELETE /jobs/job_1 to cancel.
• Put an idempotency key on the POST. Otherwise a retry creates a second job.
• When the job is done, you can either return 200 with the result inline or 303 See Other pointing at the finished resource. My toy returns a link in the body, which is the simplest option.
• Expire old jobs. Say how long the result stays available.

Polling is simple but wasteful for slow jobs. The better answer for those is the next section: the server calls the client when it is done.

Webhooks: the API that calls you

A webhook flips the direction. Instead of the client asking "is it paid yet?" every few seconds, the customer gives you a URL and you send a POST to that URL when the event happens: invoice.paid, order.shipped, user.deleted. It is the same HTTP you already know, but now you are the client and the customer's server is the API. That changes who must be careful about what.

Three problems come with it:

1. Trust. Anyone on the internet can send a POST to the customer's webhook URL, pretending to be you. How does the customer know the message is real and was not changed?
2. Delivery. The customer's server may be down, slow or buggy. You must retry, and the customer will sometimes receive the same event twice.
3. Safety of your own server. You are making requests to URLs that a customer typed in. That is a door for SSRF (a server tricked into calling internal addresses). See Security lesson 7 and the note at the end of this section.

Event design

A few simple choices make webhooks easy to consume. These follow the Standard Webhooks spec and common practice:

• Give every event a unique id, a type (invoice.paid, lower-case words with dots) and a timestamp.
• Keep the shape per event type fixed, like a REST endpoint's response. The spec compares event types to URL paths.
• Let customers choose which event types they want. Filtering on your side saves their servers.
• Decide thin or fat events. A fat event includes the full object ({"type":"invoice.paid","data":{...whole invoice...}}). A thin event only says "invoice 17 changed" and the customer calls your API to read it. Thin events are safer (the data is read fresh, with the customer's own token) but need one more call. Fat events are quicker to use but can be stale by the time they arrive.
• Do not promise order. Retries make "paid" arrive before "created" sometimes. Include a timestamp or a version number so the receiver can ignore old news.

Signing: how the receiver knows it is you

The standard answer is HMAC. You and the customer share a secret string. Before sending, you compute

signature = HMAC-SHA256( secret, id + "." + timestamp + "." + raw_body )

and put it in a header. The customer repeats the same computation. If the result matches, only someone with the secret could have made it, and the body was not changed on the way.

The Standard Webhooks spec I read uses exactly this shape. Its headers are webhook-id, webhook-timestamp (seconds since the epoch) and webhook-signature, and the signature is written as v1, followed by the base64 of the HMAC. The v1 is a version label so the scheme can change later. Different companies use their own header names and message formats (Stripe uses its own style, for example), but the idea is always the same: sign the exact bytes, include a timestamp, compare safely.

Here is a complete sender and receiver that I ran:

// hook.mjsimport { createHmac, timingSafeEqual } from "node:crypto";const SECRET = "whsec_test_secret_123";const sign = (id, ts, body, secret = SECRET) =>  "v1," + createHmac("sha256", secret).update(`${id}.${ts}.${body}`).digest("base64");function verify({ id, ts, sig, body, secret = SECRET, now = Math.floor(Date.now() / 1000), tolerance = 300 }) {  if (!id || !ts || !sig) return "missing headers";  if (Math.abs(now - Number(ts)) > tolerance) return "timestamp outside tolerance";  const expected = Buffer.from(sign(id, ts, body, secret));  for (const candidate of sig.split(" ")) {           // header may hold several signatures    const got = Buffer.from(candidate);    if (got.length === expected.length && timingSafeEqual(got, expected)) return "ok";  }  return "signature mismatch";}

Four details here matter and each one is a classic bug:

1. Sign the raw body, byte for byte. Do not parse the JSON and sign JSON.stringify(obj). Another library may write spaces or key order differently, and the signature will not match. In the test below, adding pretty-print spaces to the same JSON breaks verification. In Express this means using the raw body (express.raw()) on the webhook route before any JSON parser touches it.
2. Compare in constant time with timingSafeEqual. A normal === on strings can return faster when the first characters differ, and in theory an attacker can measure that to guess a signature one character at a time. The Standard Webhooks spec calls this out. timingSafeEqual needs two buffers of equal length, which is why we check length first.
3. Check the timestamp. The timestamp is part of what is signed. Without a time check, someone who records one real message can send it again next week (a replay attack). We reject anything more than 5 minutes (300 s) away from now.
4. Allow more than one signature. When you rotate a secret, you sign with both the old and the new key for a while. The header holds both, separated by a space, and the receiver accepts if any one matches. The last test below shows this.

Now the delivery loop. The sender tries up to four times with growing waits, and re-signs with a new timestamp on every attempt (the spec says the timestamp belongs to the attempt, which stops old attempts from being replayed):

async function deliver(event, { delays = [0, 50, 100, 200] } = {}) {  const body = JSON.stringify(event); const id = "msg_" + event.id;  for (let i = 0; i < delays.length; i++) {    await new Promise(r => setTimeout(r, delays[i]));    const ts = Math.floor(Date.now() / 1000);    const r = await fetch(target, { method: "POST", body, headers: {      "content-type": "application/json",      "webhook-id": id, "webhook-timestamp": String(ts), "webhook-signature": sign(id, ts, body) } });    console.log(`attempt ${i + 1}: receiver answered ${r.status}`);    if (r.ok) return true;  }  return false;}

The receiver reads the raw body, verifies, crashes on purpose twice (to simulate a bad deploy), and remembers each webhook-id it has handled:

const seenIds = new Set(); const handled = []; let failNext = 2;const receiver = http.createServer(async (req, res) => {  let raw = ""; for await (const c of req) raw += c;  const result = verify({ id: req.headers["webhook-id"], ts: req.headers["webhook-timestamp"],                          sig: req.headers["webhook-signature"], body: raw });  if (result !== "ok") { console.log("  receiver: rejected (" + result + ")"); res.writeHead(400); return res.end(); }  if (failNext > 0) { failNext--; console.log("  receiver: pretending to crash (500)"); res.writeHead(500); return res.end(); }  if (seenIds.has(req.headers["webhook-id"])) { console.log("  receiver: duplicate id, ack and skip"); res.writeHead(200); return res.end(); }  seenIds.add(req.headers["webhook-id"]);  handled.push(JSON.parse(raw)); console.log("  receiver: handled", raw);  res.writeHead(200); res.end();});

Real output, including a duplicate delivery of the same event and a set of attacks against verify:

--- retries with a flaky receiver  receiver: pretending to crash (500)attempt 1: receiver answered 500  receiver: pretending to crash (500)attempt 2: receiver answered 500  receiver: handled {"id":"evt_1","type":"invoice.paid","amount":4200}attempt 3: receiver answered 200--- same event delivered again (a retry after a lost 200)  receiver: duplicate id, ack and skipattempt 1: receiver answered 200--- attacks, using verify() directlyvalid:                 okbody edited:           signature mismatchre-serialised JSON:    signature mismatchwrong secret:          signature mismatchreplayed 1 hour later: timestamp outside tolerancetwo sigs (key rotation): okhandled events: 1

What this proves:

• The event was delivered after two failures, thanks to retries.
• When the same event arrived again, the receiver recognised the id and did not run the business logic twice. At the end handled events: 1. This is the same idea as an idempotency key, but now the receiver needs it. The spec suggests using webhook-id as that key. Keep ids for a while (it mentions a few minutes as an example; I would keep them longer, a day or two, because retries can span days).
• A changed body fails. A pretty-printed copy of the same JSON fails (trap 1). A wrong secret fails. A one-hour-old copy fails on time.

Sender-side operations

If you are the one sending webhooks, the spec's advice on delivery is worth copying:

• Success means any 2xx. Everything else (redirects, 4xx, 5xx, timeouts, connection errors) is a failure and gets retried. The spec says not to follow redirects, because they add load; tell the customer to update their URL.
• Retry over days, with exponential backoff and jitter. The spec's example schedule is: immediately, then after 5 seconds, 5 minutes, 30 minutes, 2 hours, 5 hours, 10 hours, 14 hours, 20 hours, and 24 hours. That is about three days in total (75 hours 35 minutes after the first try, in the spec's table).
• Stop after a long failure. Tell the customer through another channel (email), and disable the endpoint so you do not hit a dead server forever.
• Give customers a dashboard with delivery logs and a "replay this event" button. Without it, every outage becomes a support ticket.
• Set a short timeout on your own calls (a few seconds), so one slow customer does not block your sender.
• Protect yourself (SSRF). Customers give you URLs and you call them from inside your network. Refuse private IP ranges (10.x, 192.168.x, 127.x, cloud metadata addresses), do not follow redirects, and resolve the host name yourself and check the address you will actually connect to. This is the same defence as in the Security series.
• Rotate secrets with the two-signature trick shown above.

In your OpenAPI file (lesson 7) webhooks can be described too, with a top-level webhooks section. This is a feature of OpenAPI 3.1 that I checked in the spec.

Rate limiting

A rate limit says "you may send N requests per time window". There are three reasons to have one:

1. Fairness. One greedy client should not slow down everyone else.
2. Protection. Bugs, loops and attacks (password guessing, scraping) all look like too many requests.
3. Cost. Some requests cost real money: sending an SMS, calling a paid model.

OWASP lists this as API4:2023 Unrestricted Resource Consumption in the API Security Top 10. The summary says that answering requests needs network, CPU, memory and storage, and that some calls cost money per request, so a missing limit can lead to denial of service or to a large bill. OWASP also lists API6:2023 Unrestricted Access to Sensitive Business Flows: a flow like "buy a ticket" or "post a comment" can hurt the business when automated and run too often, even if no code bug exists. Limits per endpoint and per action help here, not only one global number. The login-specific side of this is in Security lesson 3.

The algorithms in plain words

Let me show the boundary problem with a fake clock, so the result is the same every time. Both limiters allow 5 requests per 10 seconds on average. Each time, a client sends 10 requests at t = 9.9 s and again 10 requests at t = 10.1 s.

// rl.mjs (the two limiters, shortened)function fixedWindow({ limit, windowSec, now }) {  const hits = new Map();  return (key) => {    const t = now(); const start = Math.floor(t / windowSec) * windowSec;    let e = hits.get(key);    if (!e || e.windowStart !== start) { e = { windowStart: start, count: 0 }; hits.set(key, e); }    e.count++;    return { allowed: e.count <= limit, remaining: Math.max(0, limit - e.count),             resetIn: Math.ceil(start + windowSec - t) };  };}function tokenBucket({ capacity, refillPerSec, now }) {  const b = new Map();  return (key) => {    const t = now(); const e = b.get(key) ?? { tokens: capacity, last: t };    e.tokens = Math.min(capacity, e.tokens + (t - e.last) * refillPerSec); e.last = t;    const allowed = e.tokens >= 1; if (allowed) e.tokens -= 1; b.set(key, e);    return { allowed, remaining: Math.floor(e.tokens),             resetIn: allowed ? 0 : Math.ceil((1 - e.tokens) / refillPerSec) };  };}

fixed window: 5 allowed at t=9.9, 5 allowed at t=10.1 -> 10 in 0.2s (limit says 5 per 10s)token bucket: 5 allowed at t=9.9, 0 allowed at t=10.1 -> 5 in 0.2s

With the fixed window, 10 requests got through in 0.2 seconds, double the promised rate, because the counter reset at t = 10. The token bucket had only refilled 0.1 tokens, so the second burst got nothing. For many APIs the fixed window is still fine, because it is so cheap. But if the limit protects something fragile, prefer a token bucket or sliding window.

What the HTTP answer should look like

When a client is over the limit, answer:

• Status 429 Too Many Requests.
• A Retry-After header: how many seconds to wait.
• A body in the standard error format from lesson 2 (problem details).
• Ideally, rate limit headers on every response, not only on the 429, so well-behaved clients can slow down before they hit the wall.

For those headers there are two worlds. Many APIs have used non-standard X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset headers for years. (I did not verify any particular provider's current docs, so check the API you integrate with.) The IETF is working on a standard. In the draft I read, there are two headers:

• RateLimit-Policy describes the rules: for example "default";q=100;w=10 means a quota (q) of 100 units per window (w) of 10 seconds. The policy name is a string you choose.
• RateLimit describes the current state: for example "default";r=50;t=30 means 50 units remaining (r) and the window resets in 30 seconds (t).

The draft also says: if a response has both RateLimit and Retry-After, then Retry-After wins. And it warns that these are hints, so clients must not trust them blindly, and servers should not leave them out of reach of abuse (a huge t would make clients stop for a very long time). Since the document is a draft, the header syntax could change before it becomes final. If you need stability today, document exactly what you send.

Here is a small server I ran that uses the fixed window, 5 requests per 10 seconds per API key, and sends the draft-style headers:

const server = http.createServer((req, res) => {  const key = req.headers["x-api-key"] ?? req.socket.remoteAddress;  const r = limiter(key);  const h = { "content-type": "application/json",    "ratelimit-policy": '"default";q=5;w=10',    "ratelimit": `"default";r=${r.remaining};t=${r.resetIn}` };  if (!r.allowed) {    res.writeHead(429, { ...h, "retry-after": String(r.resetIn), "content-type": "application/problem+json" });    return res.end(JSON.stringify({ type: "about:blank", title: "Too Many Requests", status: 429,      detail: `Limit is 5 per 10 seconds. Try again in ${r.resetIn}s.` }));  }  res.writeHead(200, h); res.end(JSON.stringify({ ok: true }));});

Client "alice" sends 7 requests, then "bob" sends one:

client 'alice' sends 7 requests:1 200 RateLimit: "default";r=4;t=1 Retry-After: -2 200 RateLimit: "default";r=3;t=1 Retry-After: -3 200 RateLimit: "default";r=2;t=1 Retry-After: -4 200 RateLimit: "default";r=1;t=1 Retry-After: -5 200 RateLimit: "default";r=0;t=1 Retry-After: -6 429 RateLimit: "default";r=0;t=1 Retry-After: 17 429 RateLimit: "default";r=0;t=1 Retry-After: 1client 'bob' first request: 200 "default";r=4;t=1429 body: {"type":"about:blank","title":"Too Many Requests","status":429,"detail":"Limit is 5 per 10 seconds. Try again in 1s."}

Two things to notice. First, r counts down 4, 3, 2, 1, 0, and request 6 is refused. Second, Bob was not affected by Alice: each key has its own counter. About the t=1: this window is aligned to the clock, and my test happened to start one second before a window boundary. On another run you will see a bigger t, up to 10. That is normal for a fixed window and a good reason to tell clients to read the header, not guess.

Decisions you must make

• What is the key? The IP address is easy but wrong for many users behind one office or phone network (they share an IP) and weak against a botnet (one attacker, many IPs). For authenticated APIs, limit by API key or user id. For anonymous endpoints like login, use IP and username. See the next lesson for how callers identify themselves.
• Different limits for different endpoints. A search call may allow 100 per minute, a "send SMS" call 5 per minute. Expensive things get small limits.
• Different plans. Free, pro, and enterprise get different quotas. The RateLimit-Policy header can tell each client its own.
• Where to enforce. Put the first line of defence at the edge: a gateway or CDN. Then add application limits for business rules ("5 password tries per hour per account").
• Many servers need shared counters. An in-memory counter on each of 4 servers allows 4 times the limit. Use a shared store such as Redis, with an atomic increment, so all servers see the same count. I did not run Redis here.
• Fail open or closed? If your limit store is down, do you let everyone through or block everyone? For most public APIs, fail open is the better trade-off (outages should not be worse), but for login and money endpoints, think twice.
• Cost-based limits. If one request can be cheap (get one item) or very expensive (export a million rows), count cost, not requests. GraphQL APIs often need this (lesson 8).
• Do not leak. A 429 should not reveal other users' data. The usual rule from lesson 2 stands: tell the client only what it needs.

Try it

1. Take the idem2.mjs server idea. Add a second endpoint POST /refunds that also needs an idempotency key. What should the key's scope be, so a payment key and a refund key never collide?
2. In the retry example, change the server so it answers 400 on the first call. How many attempts does the client make, and why?
3. A webhook receiver verifies the signature, then runs a 20-second job, then returns 200. The sender times out after 10 seconds. List two bad things that happen.
4. You have a limit of 100 requests per minute. A client sends 100 requests at 12:00:59 and 100 more at 12:01:01 with a fixed window that resets on the minute. How many got through in 2 seconds? What would a token bucket with capacity 100 and refill of 100 per minute do?
5. Your RateLimit header says t=30 but your Retry-After header says 5. Which should the client follow, based on the draft?

Answers

1. Make the key unique by combining the account, the endpoint (or operation name) and the client's key string. Store (account_id, operation, key) as the unique key. Then the same string used on two endpoints, or by two customers, will not clash.
2. One attempt. 400 is not in the retryable list, because the same request would fail again. Retrying only makes sense for temporary errors.
3. (a) The sender retries and you get the same event again while your first job is still running, so you may do the work twice unless you dedupe by webhook-id. (b) Your server holds a connection and a worker for 20 seconds per event, so under load you run out of workers and cause more timeouts. Fix: store the event, reply 200 quickly, work in the background.
4. 200 requests in 2 seconds, double the limit, because the counter reset at 12:01:00. A token bucket that is empty after the first 100 would have refilled only about 3 tokens in 2 seconds (100 per 60 s), so it lets about 3 through.
5. Retry-After takes precedence, per the draft. Wait 5 seconds. (The server may be signalling a short pause even though the full window is longer.)

Interview questions

Key takeaways

• A timeout does not mean failure. The server may have finished. Design for the lost answer.
• POST needs an idempotency key for anything that matters. Store key, fingerprint and response. Return 409 for in-progress and 422 for a changed body.
• Claim the key atomically, in the database. A check-then-save with a gap let three charges through.
• Retry only temporary errors, with backoff, jitter and Retry-After.
• Slow work becomes a job: 202 plus Location, then poll or webhook.
• Webhooks: sign the raw body with HMAC, include a timestamp, compare in constant time, accept multiple signatures for rotation, dedupe by event id, reply fast, process later.
• Rate limit by API key or user, answer 429 with Retry-After, and prefer a token bucket or sliding window when bursts matter. The IETF RateLimit headers are a draft, so document what you send.

What's next

Rate limits and webhooks both raise one question we skipped: who is calling? Next is Lesson 6: API Authentication and Authorization. We will build API keys that are stored safely, check bearer tokens, run the OAuth 2.0 authorization code flow with PKCE on a toy server, and show the number one API bug, where a user changes an id in the URL and reads someone else's data. Practise now: add an idempotency key to one POST endpoint in a project of your own, and write the test that sends it twice.