Design Patterns & LLD

Design Patterns & LLD

Design a Rate Limiter and a Notification Service (Class Models)

Token bucket and window limiters, channel strategy, backoff with jitter, and idempotency.

Lesson 0932 min read#rate-limiter#notifications#idempotency

On this page

  1. Words used in this lesson
  2. Part 1. Design a rate limiter
  3. Part 2. Design a notification service
  4. Interview questions
  5. Key takeaways
  6. What's next

In one line: a rate limiter and a notification service are two small designs that are mostly a strategy interface plus the careful details around it: for the limiter, one allow(key) method with swappable algorithms (token bucket, sliding window); for notifications, one Channel interface per way of sending (email, SMS, push), an observer to start the work, retries with backoff, and an idempotency key so nobody gets the same message twice.

This is the last class-model lesson of the series. Both questions look easy at the start ("just count requests", "just call the email API") and get interesting when you ask what if it runs twice, what if it fails, what if two requests arrive together. Those are the questions this lesson answers in code.

This lesson is the object design view. The distributed side (Redis, many servers, queues, scale) is in the system design versions: System Design, Lesson 43, Rate Limiting Algorithms Explained, Lesson 61, Design a Rate Limiter, Lesson 62, Design a Notification System, and the shorter Practical Design Problems, Lesson 2, Design a Rate Limiter and Lesson 5, Design a Notification System. For retries and idempotency in depth see System Design, Lesson 41 and Lesson 34, and Lesson 38 for dead-letter queues. The index is at /system-design. The patterns used here are in Lesson 3, and the previous lesson is Lesson 8.

What I ran / checked. All code ran on Node 22.22 with built-in type stripping, and the tests use node:test. Time, randomness and "sleeping" are all injected, so no test waits for real time. Everything is in one process and in memory: I did not run any of it against Redis, a real queue, or a real email or SMS provider, and the three channels are fakes. The algorithm names and ideas are standard, and I wrote the code myself. The numbers in the outputs (limits, waits) are small made-up examples, not production advice. The sliding window counter (a cheaper approximation) is described but I did not build it.

Words used in this lesson

  • Rate limiter decides, for each caller, whether a request may go through now or must wait.
  • Key is who is being limited: a user id, an API key or an IP address.
  • Token bucket is a bucket that holds up to N tokens. Tokens are added at a steady rate. Each request takes a token. No token, no request.
  • Backoff is waiting longer before each retry. Jitter is a random part of the wait, so many clients do not retry at the same moment.
  • Idempotency key is a unique id for one piece of work. If the same key arrives again, the system returns the first result and does not do the work again.
  • Dead letter is a message that failed every attempt and was set aside for a human or a later job.
  • Channel is a way of reaching a person: email, SMS or push.

Part 1. Design a rate limiter

Requirements: allow(key, cost = 1) returns whether the request may go ahead, how many are left, and how long to wait if not (to build a Retry-After header). Many keys, each with its own count. The algorithm must be swappable. I assume one process first, and discuss many servers after.

The class design is tiny, and that is the point.

RateLimiter is the interface, Decision is the answer, and TokenBucket, FixedWindow and SlidingWindowLog are strategies with the same interface. The clock is injected.

The caller (an HTTP middleware, say) depends only on RateLimiter. Changing the algorithm, or using a different one for each plan, changes no caller. That is the strategy pattern and the dependency inversion principle from Lesson 0.

JavaScript
export interface Decision {  allowed: boolean;  remaining: number;  retryAfterMs: number; // 0 when allowed}export interface RateLimiter {  allow(key: string, cost?: number): Decision;}// Token bucket: tokens refill at a steady rate, up to a maximum. A request spends tokens.export class TokenBucket implements RateLimiter {  private readonly buckets = new Map<string, { tokens: number; at: number }>();  private readonly capacity: number;  private readonly refillPerSec: number;  private readonly now: () => number;  constructor(capacity: number, refillPerSec: number, now: () => number = Date.now) {    this.capacity = capacity;    this.refillPerSec = refillPerSec;    this.now = now;  }  allow(key: string, cost = 1): Decision {    const t = this.now();    const b = this.buckets.get(key) ?? { tokens: this.capacity, at: t };    // Refill lazily: work out what we earned since the last call. No timers needed.    b.tokens = Math.min(this.capacity, b.tokens + ((t - b.at) / 1000) * this.refillPerSec);    b.at = t;    this.buckets.set(key, b);    if (b.tokens >= cost) {      b.tokens -= cost;      return { allowed: true, remaining: Math.floor(b.tokens), retryAfterMs: 0 };    }    const missing = cost - b.tokens;    return { allowed: false, remaining: Math.floor(b.tokens), retryAfterMs: Math.ceil((missing / this.refillPerSec) * 1000) };  }}// Fixed window: one counter per time slot. Cheap, but allows a burst across a slot boundary.export class FixedWindow implements RateLimiter {  private readonly counts = new Map<string, { slot: number; n: number }>();  private readonly limit: number;  private readonly windowMs: number;  private readonly now: () => number;  constructor(limit: number, windowMs: number, now: () => number = Date.now) {    this.limit = limit;    this.windowMs = windowMs;    this.now = now;  }  allow(key: string, cost = 1): Decision {    const t = this.now();    const slot = Math.floor(t / this.windowMs);    let c = this.counts.get(key);    if (!c || c.slot !== slot) c = { slot, n: 0 };    this.counts.set(key, c);    if (c.n + cost <= this.limit) {      c.n += cost;      return { allowed: true, remaining: this.limit - c.n, retryAfterMs: 0 };    }    return { allowed: false, remaining: this.limit - c.n, retryAfterMs: (slot + 1) * this.windowMs - t };  }}// Sliding window log: remember each request time; count those in the last windowMs. Exact, but O(limit) memory per key.export class SlidingWindowLog implements RateLimiter {  private readonly logs = new Map<string, number[]>();  private readonly limit: number;  private readonly windowMs: number;  private readonly now: () => number;  constructor(limit: number, windowMs: number, now: () => number = Date.now) {    this.limit = limit;    this.windowMs = windowMs;    this.now = now;  }  allow(key: string, cost = 1): Decision {    const t = this.now();    const log = (this.logs.get(key) ?? []).filter((x) => x > t - this.windowMs); // drop old entries    this.logs.set(key, log);    if (log.length + cost <= this.limit) {      for (let i = 0; i < cost; i++) log.push(t);      return { allowed: true, remaining: this.limit - log.length, retryAfterMs: 0 };    }    // The oldest entry that must leave the window before `cost` more fit:    const mustExpire = log[log.length + cost - this.limit - 1];    return { allowed: false, remaining: this.limit - log.length, retryAfterMs: mustExpire + this.windowMs - t };  }}

The three algorithms compared

AlgorithmState per keyAllows bursts?Weak spot
Token bucket2 numbers (tokens, last time)Yes, up to the bucket size, then the steady rateA big bucket allows a big burst
Fixed window1 counter and a slot numberUp to 2x the limit across a boundaryThe boundary burst, below
Sliding window logA list of timestamps (up to the limit)No: exactMemory grows with the limit
Sliding window counter (not built)2 countersApproximately noIt is an estimate, not exact

Notes on the code:

  • Token refill is lazy. There is no timer. On each call I compute how many tokens were earned since the last call. This keeps the limiter free of background work, and it is easy to test.
  • retryAfterMs is computed, not guessed. For the bucket it is missing tokens / refill rate. For the log it is when the oldest counted request leaves the window. For the fixed window it is the time to the next slot.
  • cost lets one expensive call (say a report) spend several tokens.

Here is the boundary problem, and the bucket refilling:

JavaScript
import { FixedWindow, SlidingWindowLog, TokenBucket, type RateLimiter } from "./limiters.ts";// 5 requests per 10 seconds. Send 5 at t=9.9s and 5 more at t=10.1s.function burst(name: string, make: (now: () => number) => RateLimiter) {  let t = 0;  const l = make(() => t);  let ok = 0;  for (const at of [9900, 10100]) { t = at; for (let i = 0; i < 5; i++) if (l.allow("u1").allowed) ok++; }  console.log(name.padEnd(18), "allowed", ok, "of 10 requests in 0.2 s");}burst("fixed window", (now) => new FixedWindow(5, 10_000, now));burst("sliding log", (now) => new SlidingWindowLog(5, 10_000, now));burst("token bucket", (now) => new TokenBucket(5, 0.5, now)); // 5 capacity, refill 1 token per 2 slet t = 0;const tb = new TokenBucket(3, 1, () => t); // capacity 3, 1 token per secondconst row = () => { const d = tb.allow("u1"); return `${d.allowed ? "ok  " : "deny"} left=${d.remaining} retryAfter=${d.retryAfterMs}ms`; };for (t of [0, 0, 0, 0]) console.log(`t=${t}ms`, row());t = 1000; console.log(`t=${t}ms`, row());t = 1500; console.log(`t=${t}ms`, row());t = 60_000; console.log(`t=${t}ms`, row(), "(bucket refilled, but only up to capacity)");
TEXT
$ node demo.tsfixed window       allowed 10 of 10 requests in 0.2 ssliding log        allowed 5 of 10 requests in 0.2 stoken bucket       allowed 5 of 10 requests in 0.2 st=0ms ok   left=2 retryAfter=0mst=0ms ok   left=1 retryAfter=0mst=0ms ok   left=0 retryAfter=0mst=0ms deny left=0 retryAfter=1000mst=1000ms ok   left=0 retryAfter=0mst=1500ms deny left=0 retryAfter=500mst=60000ms ok   left=2 retryAfter=0ms (bucket refilled, but only up to capacity)

Read the first three lines. The limit is 5 per 10 seconds. I sent 5 requests at 9.9 s and 5 more at 10.1 s. The fixed window let all 10 through in 0.2 seconds, because the clock crossed into a new slot and the counter reset. The sliding log and the token bucket let 5 through. That is the standard weakness of fixed windows.

Then the bucket table: capacity 3, one token per second. Three requests pass at once (a burst), the fourth is refused with retryAfter=1000ms, and one second later one passes. At 1.5 s only half a token has come back, so it is refused with 500 ms to wait. After a full minute the bucket is full again but not more than 3 (left=2 after spending one).

From one process to many servers

The classes above keep counts in a Map in one process. With several servers behind a load balancer, each server would count only its own share. So the state must move to one shared place, and the check-and-update must be one atomic step (the same compare-and-set idea as in Lesson 5, where I showed a double booking). I have not run a shared-store version here; see the system design lessons above for Redis scripts and the trade-offs of failing open or closed when the store is down.

Other class-level questions:

They askShort answer
Different limits per plan or routeA small registry: plan -> RateLimiter. A factory builds the right one.
Memory leak from many keysRemove idle keys: a bucket that is full and untouched is the same as no bucket, so it can be deleted.
Limit by user and by IP togetherCheck both limiters. Only count the request if both allow it, or you consume tokens for refused requests.
What to send when refusedHTTP 429 and a Retry-After value. retryAfterMs is the source.

Part 2. Design a notification service

Requirements: other services publish events ("order shipped"). The service looks up the user's preferences, builds the text from a template, sends it on each enabled channel (email, SMS, push), retries temporary failures, never sends the same message twice, and keeps failures somewhere visible.

PiecePatternWhy
EventBusObserverProducers publish and do not know who listens
Channel per mediumStrategyAdd Slack by adding one class
templatesDataText changes without code changes
withRetryHelper functionOne place for backoff rules
done map keyed by eventId:channelIdempotencyDuplicate events are harmless
deadLettersDead-letter listFailures are kept, not lost

Key decisions:

  • The idempotency key is per event and per channel, not per event. If email succeeded and push failed, a redelivery of the same event must resend only the push. A per-event key would either resend the email or skip the push.
  • The map stores the promise, not the result. Two copies of the same event arriving at the same moment share one send. This is the same trick as the in-flight load in Lesson 6.
  • A failed delivery removes its key, so a later redelivery may try again. A sent delivery keeps its key.
  • Permanent and temporary errors are different. A timeout may work next time. An invalid address never will, so PermanentError stops the retries at once.
  • Backoff with full jitter: the wait before attempt n is a random number between 0 and min(cap, base * 2^n). Jitter spreads out the retries of many clients so they do not all hit a recovering service at the same moment.
JavaScript
export type ChannelName = "email" | "sms" | "push";export type Prefs = Record<ChannelName, boolean>;export interface DomainEvent { id: string; type: string; userId: string; data: Record<string, string> }export interface Notification { to: string; channel: ChannelName; subject: string; body: string; key: string }// Strategy: one interface, one class per channel. Adding "slack" means adding a class.export interface Channel {  readonly name: ChannelName;  send(n: Notification): Promise<void>;}// A failure that retrying cannot fix (bad address). The retry helper stops at once.export class PermanentError extends Error {}export interface RetryOptions {  attempts: number;  baseMs: number;  capMs: number;  sleep: (ms: number) => Promise<void>;  rand: () => number; // 0..1, injected}// Exponential backoff with full jitter: wait a random time between 0 and min(cap, base * 2^n).export async function withRetry<T>(fn: () => Promise<T>, o: RetryOptions): Promise<{ value: T; attempts: number }> {  for (let attempt = 1; ; attempt++) {    try {      return { value: await fn(), attempts: attempt };    } catch (e) {      if (e instanceof PermanentError || attempt >= o.attempts) throw Object.assign(e as Error, { attempts: attempt });      await o.sleep(o.rand() * Math.min(o.capMs, o.baseMs * 2 ** (attempt - 1)));    }  }}export type Delivery = { channel: ChannelName; status: "sent" | "failed"; attempts: number; error?: string };export interface Deps {  channels: Channel[];  prefs: (userId: string) => Prefs;  templates: Record<string, { subject: string; body: string }>;  retry: RetryOptions;}export class NotificationService {  readonly deadLetters: { notification: Notification; error: string }[] = [];  private readonly done = new Map<string, Promise<Delivery>>(); // idempotency: key -> result  private readonly d: Deps;  constructor(deps: Deps) { this.d = deps; }  private render(e: DomainEvent) {    const t = this.d.templates[e.type];    if (!t) throw new Error(`no template for ${e.type}`);    const fill = (s: string) => s.replace(/\{(\w+)\}/g, (_, k) => e.data[k] ?? "");    return { subject: fill(t.subject), body: fill(t.body) };  }  async handle(e: DomainEvent): Promise<Delivery[]> {    const prefs = this.d.prefs(e.userId);    const text = this.render(e);    const jobs = this.d.channels.filter((c) => prefs[c.name]).map((c) => {      const key = `${e.id}:${c.name}`; // one key per event and channel      const existing = this.done.get(key);      if (existing) return existing; // already sent, or being sent right now      const n: Notification = { to: e.userId, channel: c.name, ...text, key };      const p = this.deliver(c, n).then((r) => {        if (r.status === "failed") this.done.delete(key); // a later redelivery may try again        return r;      });      this.done.set(key, p);      return p;    });    return Promise.all(jobs);  }  private async deliver(c: Channel, n: Notification): Promise<Delivery> {    try {      const { attempts } = await withRetry(() => c.send(n), this.d.retry);      return { channel: c.name, status: "sent", attempts };    } catch (e) {      this.deadLetters.push({ notification: n, error: (e as Error).message });      return { channel: c.name, status: "failed", attempts: (e as { attempts?: number }).attempts ?? 1, error: (e as Error).message };    }  }}// Observer: services publish events and do not know who listens.export class EventBus {  private readonly handlers = new Map<string, ((e: DomainEvent) => Promise<unknown>)[]>();  on(type: string, h: (e: DomainEvent) => Promise<unknown>): void {    this.handlers.set(type, [...(this.handlers.get(type) ?? []), h]);  }  async publish(e: DomainEvent): Promise<void> {    await Promise.allSettled((this.handlers.get(e.type) ?? []).map((h) => h(e)));  }}

Running it

In the demo, email times out twice and then works, SMS has an invalid address, and push always works. I fixed rand to 1 so the waits are the full ceiling and the output is the same every time.

JavaScript
import { EventBus, NotificationService, PermanentError, type Channel, type ChannelName, type Notification } from "./notify.ts";const line = (ds: { channel: string; status: string; attempts: number }[]) => ds.map((d) => `${d.channel}:${d.status}(${d.attempts} tries)`).join("  ");const sleeps: number[] = [];function fake(name: ChannelName, plan: ("ok" | "fail" | "bad")[]): Channel & { sent: string[] } {  const sent: string[] = [];  return {    name, sent,    async send(n: Notification) {      const step = plan.shift() ?? "ok";      if (step === "fail") throw new Error(`${name} timeout`);      if (step === "bad") throw new PermanentError(`${name} address invalid`);      sent.push(`${n.key} "${n.subject}"`);    },  };}const email = fake("email", ["fail", "fail"]); // fails twice, then worksconst sms = fake("sms", ["bad"]);              // never worksconst push = fake("push", []);                 // always worksconst svc = new NotificationService({  channels: [email, sms, push],  prefs: () => ({ email: true, sms: true, push: true }),  templates: { "order.shipped": { subject: "Order {order} shipped", body: "Hi {name}, order {order} is on its way." } },  retry: { attempts: 3, baseMs: 100, capMs: 1000, sleep: async (ms) => { sleeps.push(ms); }, rand: () => 1 }, // rand=1: the longest wait});const bus = new EventBus();bus.on("order.shipped", (e) => svc.handle(e));const event = { id: "evt-1", type: "order.shipped", userId: "u1", data: { order: "#42", name: "Ann" } };console.log("first :", line(await svc.handle(event)));console.log("waits :", sleeps.join(", "), "ms (rand fixed at 1, so each wait is its full ceiling)");console.log("email :", email.sent, "| push:", push.sent);console.log("again :", line(await svc.handle(event))); // same event id: only the failed channel is tried againconsole.log("email :", email.sent.length, "sent in total | push:", push.sent.length, "| dead letters:", svc.deadLetters.length);await bus.publish({ ...event, id: "evt-2" });console.log("after a new event through the bus, push sent:", push.sent.length);
TEXT
$ node notify-demo.tsfirst : email:sent(3 tries)  sms:failed(1 tries)  push:sent(1 tries)waits : 100, 200 ms (rand fixed at 1, so each wait is its full ceiling)email : [ 'evt-1:email "Order #42 shipped"' ] | push: [ 'evt-1:push "Order #42 shipped"' ]again : email:sent(3 tries)  sms:sent(1 tries)  push:sent(1 tries)email : 1 sent in total | push: 1 | dead letters: 1after a new event through the bus, push sent: 2

Read it line by line:

  1. first: email needed 3 tries (it failed twice, then worked), SMS failed after 1 try (permanent error, no retry), push worked at once.
  2. waits: 100 ms then 200 ms, the ceilings for attempts 1 and 2 (base 100, doubling).
  3. Only one email and one push were recorded as sent.
  4. again: the same event id is handled again. Email and push were not sent again (their keys are stored). SMS was tried again because it had failed, and this time it worked only because my fake channel fails just once. In real life an invalid address keeps failing.
  5. Totals: still 1 email and 1 push sent, 1 dead letter kept.
  6. A new event (evt-2) published through the bus does reach push, so the total is 2.

Tests

JavaScript
const opts = (sleeps: number[], rand = () => 1) => ({ attempts: 4, baseMs: 100, capMs: 500, rand, sleep: async (ms: number) => { sleeps.push(ms); } });test("retry: exponential waits, capped, with jitter scaling them down", async () => {  const sleeps: number[] = [];  await assert.rejects(withRetry(async () => { throw new Error("x"); }, opts(sleeps)), /x/);  assert.deepEqual(sleeps, [100, 200, 400]); // 4 attempts means 3 waits  const half: number[] = [];  await assert.rejects(withRetry(async () => { throw new Error("x"); }, { ...opts(half, () => 0.5), attempts: 5 }));  assert.deepEqual(half, [50, 100, 200, 250]); // 800 was capped to 500 first, then halved});
JavaScript
test("notify: the same event is never sent twice, even when two arrive at once", async () => {  const e = chan("email");  const svc = service([e.c]);  await Promise.all([svc.handle(ev("1")), svc.handle(ev("1"))]);  await svc.handle(ev("1"));  assert.equal(e.sent.length, 1);});test("notify: a failed channel goes to dead letters and is tried again on redelivery; sent channels are not", async () => {  const e = chan("email"), p = chan("push", 2); // push fails both attempts the first time  const svc = service([e.c, p.c]);  const first = await svc.handle(ev("1"));  assert.deepEqual(first.map((d) => d.status), ["sent", "failed"]);  assert.equal(svc.deadLetters.length, 1);  const second = await svc.handle(ev("1"));  assert.deepEqual(second.map((d) => d.status), ["sent", "sent"]);  assert.deepEqual([e.sent.length, p.sent.length], [1, 1]);});
TEXT
$ node --test --test-reporter=spec limiter.test.ts | LC_ALL=C sed 's/\xE2\x9C\x94/pass/; s/\xE2\x84\xB9/#/'pass token bucket: burst up to capacity, then wait for refill (4.142396ms)pass token bucket: refill never goes above capacity, and a cost can be more than 1 (0.499571ms)pass keys are separate (0.470908ms)pass fixed window lets a double burst through at the boundary; sliding log does not (1.105441ms)pass sliding log: retryAfter says when the oldest request leaves the window (0.369778ms)pass retry: exponential waits, capped, with jitter scaling them down (2.129291ms)pass retry: a permanent error is not retried (0.753172ms)pass notify: only channels the user enabled are used (1.100895ms)pass notify: the same event is never sent twice, even when two arrive at once (1.101863ms)pass notify: a failed channel goes to dead letters and is tried again on redelivery; sent channels are not (1.519867ms)pass event bus: one failing handler does not stop the others (0.733341ms)# tests 11# suites 0# pass 11# fail 0# cancelled 0# skipped 0# todo 0# duration_ms 283.011835

What it does not solve

The idempotency map here lives in memory, so after a restart the service forgets what it sent. A real service stores the keys in a database or Redis, with an expiry, and inserts the key before sending with a unique constraint, so two servers cannot both pass. There is still a window between "sent" and "recorded": if the process dies in between, the message may be sent twice. This is the usual at least once delivery trade-off. Which is worse, a duplicate or a lost message, is a product decision; ask. I did not simulate this crash.

Interview questions

Design a rate limiter's classes. (Must know)

One interface, allow(key, cost) returning allowed, remaining and retry-after. Each algorithm is a class that implements it, and the clock is injected. Callers depend only on the interface, so I can change the algorithm or choose one per plan without touching them.

Explain the token bucket. (Must know)

A bucket holds up to N tokens and refills at a fixed rate. Each request takes a token. If the bucket is empty, the request is refused. It allows a burst of up to N, then the steady refill rate. I refill lazily on each call from the elapsed time, so there are no timers. It stores two numbers per key.

Fixed window or sliding window? (Must know)

Fixed window is the cheapest, one counter, but it lets through up to twice the limit around a boundary: in my run, 10 requests in 0.2 s when the limit was 5 per 10 s. A sliding log is exact but keeps one timestamp per request. A sliding counter is a cheaper estimate. I pick by how exact the limit must be and how much memory I can spend per key.

How do you make it work on many servers? (Senior signal)

Move the counts to one shared store and make the read-and-update atomic, otherwise two servers both see "one token left". Then decide what happens if the store is down: fail open (let traffic through) or fail closed (block). That is a risk decision, so I would ask about it. I have not run this in the lesson; the design is in the system design series.

Why a Channel interface? (Must know)

Each medium has its own client, errors and limits, but the service only needs "send this". With an interface, adding Slack is one new class and no change to the service. It also lets tests use fakes instead of real providers.

Where do you use the observer pattern here? (Must know)

Between the services that produce events and the notification service. The order service publishes order.shipped and does not know who listens. The notification service subscribes. Later, an analytics handler can subscribe too, without any change to the order code. In my event bus, one failing handler does not stop the others (tested).

How do you stop duplicate notifications? (Must know)

An idempotency key per event and channel, stored with the result. A second request with the same key returns the first result and sends nothing. I store the promise so that two copies arriving at the same instant also share one send. In production, the keys live in a shared store with an expiry, and the insert is the guard.

Why per channel and not per event? (Trap)

With one key per event, a partial failure is stuck: either the successful email is sent again on redelivery, or the failed push is skipped because the event is "done". A key per event:channel lets me resend only what failed.

How do you design the retries? (Must know)

Exponential backoff with jitter and a cap, a maximum number of attempts, and a list of errors that are worth retrying. The wait before attempt n is random between 0 and min(cap, base * 2^n). Permanent errors, like a bad address, are not retried. After the last attempt the notification goes to a dead-letter list. I also keep retries off the request path: the service should run them in the background.

Why jitter? (Senior signal)

Without jitter, all clients that failed together wait the same time and retry together, which can knock a recovering service down again. Random waits spread the load out. I tested the formula with a fixed rand: with rand = 0.5 the waits were half the capped ceiling.

What happens if the process crashes after sending but before recording? (Senior signal)

The message may be sent again after restart. That is at-least-once delivery. To get closer to exactly once, you need the provider to support idempotency keys too, and you pass your key along. Otherwise you accept the rare duplicate, or you accept the rare loss by recording first. I would say which one the product prefers: for a password reset a duplicate is fine, for charging a card it is not.

Key takeaways

  • A rate limiter is one interface with swappable algorithms. Inject the clock.
  • Token bucket: bursts then a steady rate, lazy refill, two numbers per key. Fixed window: cheap but can allow double the limit at a boundary. Sliding log: exact, more memory.
  • Return retryAfterMs from allow, so the HTTP layer can send 429 and a Retry-After.
  • Many servers need shared state and an atomic update.
  • Notification service: observer for events, strategy per channel, templates as data.
  • Retry with exponential backoff, a cap and jitter. Never retry permanent errors. Keep failures in a dead-letter list.
  • Idempotency key per event and channel. Store the promise to cover concurrent duplicates.
  • In memory is a first answer. In production the keys and counts go to a shared store.

What's next

This finishes the class-model lessons. Go back to Lesson 0 and pick a question you have not done. Do it from a blank page in 35 minutes: ask questions, list the nouns, draw the classes, write a test first, and say your trade-offs out loud. Practise: connect the two parts of this lesson by rate limiting notifications per user, and write the tests first.

Published by Jainik · Last updated · How these lessons are made

Keep learning

Related lessons

Part of the study guide

Related series