Design a Unique ID Generator

This is lesson 6 of System Design Case Studies, Part 2. In Lesson 1 we partitioned data by key hash. In Lesson 5 the queue, the shards and the time buckets all needed names that never repeat. This lesson builds those names. If you want the basics of replication and sharding first, see System Design Lesson 26 (replication). Later, Lesson 8 (matching engine) needs strictly ordered IDs for orders, which is a stronger demand than we meet here.

Words used in this lesson

• Unique ID is a value that identifies one record (a user, an order, a message). Two records must never share one.
• Auto-increment is a database counter that gives 1, 2, 3... to each new row.
• Primary key is the column that identifies a row. The database keeps an index (a sorted tree) on it.
• UUID (universally unique identifier) is a 128-bit ID, usually written as 36 characters: 8-4-4-4-12 hex digits.
• Epoch is the "time zero" that a timestamp counts from. Unix time counts milliseconds from 1970-01-01. A custom epoch starts later, to save bits.
• Snowflake is the ID scheme from Twitter: time, machine number and counter packed into 64 bits.
• Sequence is the counter that separates IDs made in the same millisecond on the same machine.
• k-sorted (roughly sorted) means IDs are in time order within a small bound, not exactly.
• Clock skew is when two machines disagree on the time. Clock step back is when one machine's clock jumps backwards (for example after a time sync).
• NTP (network time protocol) is the service that keeps a machine's clock close to real time.
• Birthday bound is the number of random values you need before a repeat becomes likely. It is about the square root of the number of possible values.
• Ticket server is a small database whose only job is to hand out the next number.

1. Requirements

Functional

• next() returns a new ID that has never been returned before, anywhere in the system.
• Many services and machines call it at the same time, in several data centres.
• Fit in 64 bits if possible (a BIGINT), because the ID is stored in every table, index and foreign key.
• IDs sort roughly by creation time, so "give me items since ID X" and "newest first" work without a separate timestamp column.

Non-functional

• No coordination on the hot path: a machine should make IDs without calling another machine.
• Fast: microseconds per ID, not milliseconds.
• Available: ID generation must not be the single point of failure (System Design Lesson 40).
• Safe under bad clocks: never issue a duplicate, even if the clock misbehaves. Stopping is better than duplicating.
• Length and format: easy to store, to log and to put in a URL.

Out of scope here: strictly gap-free numbers (invoice numbers that the law says must be consecutive), and IDs that must be secret. Time-ordered IDs leak when they were made, and sequence counters leak volume.

2. Back-of-envelope numbers

Every input is an assumption unless it comes from the source named.

Assume       200,000 new IDs per second at peak, across the whole company             -> 200,000 x 86,400 x 365 = 6.3e12 IDs per year (6.3 trillion)INT (32 bit) signed max      2,147,483,647      -> gone in 2,147,483,647 / 200,000 = 10,737 s = 3 hoursINT unsigned max             4,294,967,295      -> 6 hoursBIGINT signed max            9,223,372,036,854,775,807 (2^63 - 1)                             -> 9.2e18 / 6.3e12 per year = 1.46 million years   (for a plain counter)

So the first rule of this lesson: a 32-bit auto-increment column is a time bomb at scale, and a 64-bit one is not. The hard part with a counter is not the size. It is that one counter means one machine to ask, and one machine to fail.

Now the same maths for the Snowflake layout. I ran this in ids.ts:

== bit maths ==2^41 ms          = 2199023255552 ms = 69.68 years2^12 per ms      = 4096 ids per ms per worker = 4096000 ids/s per worker2^10 workers     = 1024 -> cluster max = 4194304 ids/ms = 4,194,304,000 ids/stotal bits       = 64max id (63 bits) = 9223372036854775807 | JS safe integer limit = 9007199254740991

Read it like this:

• Time: 41 bits of milliseconds is 69.68 years. Twitter's README says the same: "41 bits (millisecond precision w/ a custom epoch gives us 69 years)". A custom epoch (we use 2024-01-01) means the 69 years start then, not in 1970. With a 1970 epoch, 41 bits would run out in 2039.
• Machine: 10 bits gives 1,024 workers. (The README says "configured machine id - 10 bits - gives us up to 1024 machines". Splitting it 5 + 5 as datacentre + worker is a common choice from memory, not a rule.)
• Sequence: 12 bits gives 4,096 IDs per millisecond per worker, which is 4.096 million per second. We need 200,000 per second in total, so even 100 workers have a huge margin.
• Sign bit: the top bit is left at 0, so the ID is a positive signed 64-bit number. Many languages and databases use signed 64-bit.

3. API

# Option A: a library in each service (preferred: no network on the hot path)const id = idgen.next();            // returns a 64-bit integer, no network call# Option B: a small service (when many languages need it)POST /v1/ids            { "count": 100 }→ 200 OK   { "ids": ["367651388534493184", ...] }       # strings, not numbers (see 6.7)GET  /v1/ids/367651388534493184/decode→ { "createdAt": "2026-10-11T12:35:21.659Z", "dc": 3, "worker": 17, "seq": 0 }

Notes:

• IDs travel as strings in JSON. I show why in section 6.7.
• A decode endpoint is for debugging only. Business logic must not read meaning from ID bits, or you cannot change the layout later.
• Batching (count: 100) saves round trips when you use a service.

4. Data model

The generator itself keeps almost nothing:

Per worker, in memory:   lastMs (the millisecond of the last ID), seq (the counter in that millisecond)Worker number:           leased from a small coordination store (ZooKeeper, etcd or a database row)                         when the process starts, with a lease that expires if the process dies.

The ID layout (63 bits used, 64 bits wide):

 bit 63      62 ................... 22   21 .. 17    16 .. 12    11 ........ 0 +------+---------------------------+------------+------------+--------------+ |  0   |   milliseconds since      | datacentre |   worker   |   sequence   | | sign |   our epoch (41 bits)     |  (5 bits)  |  (5 bits)  |  (12 bits)   | +------+---------------------------+------------+------------+--------------+ shifts:     time << 22                dc << 17      worker << 12     seq << 0

The time is the most significant part, so comparing IDs as numbers compares creation time first. That is the whole reason IDs sort by time.

5. High-level architecture

   Service A (many pods)            Service B (many pods)           Batch job   [ idgen library ]                [ idgen library ]               [ idgen library ]        |   worker = 17                  worker = 18                     worker = 40        |   next() in memory             next() in memory                next() in memory        v                                v                               v     INSERT row with id=...          INSERT row with id=...          INSERT row with id=...                    \                        |                        /                     +--------- sharded database / queue / logs -----+   at process start only:   idgen library --lease a worker number (TTL 30 s, renew every 10 s)--> Coordination store (etcd / ZooKeeper / DB)                   <-- "you are dc 3, worker 17" ---------------------------   always:  clock source -- NTP / chrony --> local clock (must not step backwards)

The hot path (next()) is pure memory and the local clock. The only shared thing is the worker number, and it is needed only at startup. If the coordination store is down, running processes keep working. Only new processes cannot start.

6. Deep dives

6.1 Auto-increment: simple, and where it stops

id BIGINT AUTO_INCREMENT (MySQL) or GENERATED ... AS IDENTITY (PostgreSQL) is the right answer for one database. It is gap-friendly, tiny and sorted. It breaks when:

• There is more than one writer. Two primaries both counting from 1 will duplicate. The old trick is "odd and even": server A uses 1, 3, 5 and server B uses 2, 4, 6 (MySQL has auto_increment_increment and auto_increment_offset for this, from memory). It works for 2 servers and becomes painful when you add a third.
• You shard. Each shard counts on its own, so IDs collide across shards unless you add the shard number somewhere.
• The ID leaks business facts. /orders/1042 tells a competitor how many orders you have. Counting two numbers a week apart tells them your growth.
• A 32-bit column fills up. See section 2: 3 to 6 hours at 200,000 per second.
• Gaps happen anyway. In PostgreSQL a rolled-back insert still uses up a sequence value (from memory). Do not promise "no gaps" with a sequence.

6.2 UUID v4: unique with no coordination, bad for indexes

A UUID has 128 bits. Version 4 fills 122 of them with random bits (the other 6 are the version and variant markers). Machines need no coordination at all. How safe is it? The birthday bound, which I computed:

== birthday bound: ids needed for a 50% chance of one collision == 64 random bits -> about 5.06e+9 ids 80 random bits -> about 1.29e+12 ids122 random bits -> about 2.71e+18 ids1e9 UUIDv4 ids: p(collision) ~ 9.40e-20

With 122 random bits you would need about 2.7e18 IDs for a coin-flip chance of one repeat. A billion IDs have a collision chance near 1e-19, so in practice the risk is zero if the random numbers are good (use the operating system's secure random source; crypto.randomBytes in Node). The 64 random bits line is a warning: random 64-bit IDs are not safe at billions of rows, because a repeat becomes likely after about 5 billion.

The real cost of v4 is not collisions. It is storage and index behaviour. The draft says it directly: "Non-time-ordered UUID versions such as UUIDv4 ... have poor database index locality", and the negative effect on B-trees "can be dramatic". The idea: each new key lands in a random place of the B-tree index (System Design Lesson 22). Pages all over the tree get touched, the working set of the index must stay in memory, and pages split often. IDs that grow with time all write to the right edge of the tree, which stays hot and small. Also v4 is 16 bytes, twice a BIGINT, and in every secondary index too.

6.3 UUID v7: time first, then random

UUIDv7 keeps the 128 bits and the text format, but puts a 48-bit Unix millisecond timestamp first. I checked the layout in the draft's text:

unix_ts_ms  48 bits   bits 0-47   (big-endian milliseconds since 1970)ver          4 bits   bits 48-51  (0b0111 = 7)rand_a      12 bits   bits 52-63  (random, or an optional counter)var          2 bits   bits 64-65  (0b10)rand_b      62 bits   bits 66-127 (random, or an optional counter)total      128 bits   = 48 + 4 + 12 + 2 + 62

The draft also gives a test vector: timestamp 0x017F22E279B0 (1645557742000 ms), rand_a = 0xCC3, rand_b starting 0b01, 0x8C4DC0C0C07398F, final 017F22E2-79B0-7CC3-98C4-DC0C0C07398F. I wrote a 15-line uuidv7() and fed it the same inputs:

== UUID v4 / v7 ==v4 sample: 571d5469-9f22-41aa-91f2-f7afb85b68ecv7 sample: 01a12af5-e9b9-77ef-88a1-d3ab5ab3a984spec vector expected: 017f22e2-79b0-7cc3-98c4-dc0c0c07398four uuidv7 produced : 017f22e2-79b0-7cc3-98c4-dc0c0c07398f (match)v7 sorted as text == creation order: truev4 sorted as text == creation order: false (usually false)

Look at the v4 sample: the 13th hex digit is 4 (version) and the 17th is 9 (one of 8, 9, a, b: the variant). In the v7 sample the 13th digit is 7 and the 17th is 8. v7 text sorts in creation order (at millisecond resolution), v4 does not.

The 74 random bits per millisecond are plenty for uniqueness, but v7 values made in the same millisecond are not ordered among themselves unless you add a counter. The draft allows this ("an OPTIONAL carefully seeded counter") in rand_a/rand_b. My simple version does not do that.

When do I pick UUIDv7? When I want no machine numbers, no coordination and a 128-bit key is fine: for example IDs created by clients (mobile apps creating records offline), or by many unrelated services. When I need a compact 64-bit key, or the system already runs a worker-number lease, I pick Snowflake.

6.4 ULID

ULID (universally unique lexicographically sortable identifier) is the same idea as v7 with a different text form. From the ulid/spec README: 128 bits, a 48-bit timestamp (good "'til the year 10889 AD") plus 80 bits of randomness, written as 26 characters of Crockford's base32 (an alphabet without I, L, O, U), case-insensitive. The spec has a monotonic mode: if two ULIDs are made in the same millisecond, the random part is incremented by one, and the spec says generation fails if that overflows. I wrote a small encoder (no monotonic mode):

== ULID ==ulid: 01FWHE4YDGSFTEYWX5TVHQ8RB9 length 26 | time part: 01FWHE4YDG

(I passed the timestamp 1645557742000, the same as the UUID vector. The first 10 characters are the time, the last 16 the randomness.) ULID is nicer in logs and URLs than a 36-character UUID. It does not carry UUID version bits, so some tools that validate UUIDs reject it.

6.5 Snowflake: the bit maths and the code

The layout is in section 4. The code, from snowflake.ts, is 72 lines. The core:

next(): bigint {  let ts = BigInt(this.now());  if (ts < this.lastMs) {                       // the clock went backwards    const behind = Number(this.lastMs - ts);    if (behind > this.maxWait) throw new ClockMovedBackError(`clock is ${behind} ms behind the last id`);    this.sleep(behind);                         // small step back: wait it out    ts = BigInt(this.now());    if (ts < this.lastMs) throw new ClockMovedBackError("clock still behind after waiting");  }  if (ts === this.lastMs) {    this.seq = (this.seq + 1n) & MAX_SEQ;       // MAX_SEQ = 4095    if (this.seq === 0n) {                      // 4096 ids used up in this millisecond      this.overflowWaits++;      do { ts = BigInt(this.now()); } while (ts <= this.lastMs); // spin until the next ms    }  } else {    this.seq = 0n;  }  this.lastMs = ts;  const t = ts - EPOCH_MS;  return (t << TIME_SHIFT) | (this.dc << DC_SHIFT) | (this.worker << WORKER_SHIFT) | this.seq;}

Why BigInt: a JavaScript number is a 64-bit float with only 53 bits of whole-number precision. Our IDs have 63 bits. (The demo prints the safe limit, 9007199254740991.) Use BigInt in JavaScript, long in Java, int64 in Go.

One ID, shown in binary and decoded:

== Snowflake ==id        : 367651388534493184 | binary: 000010100011010001010001111110101101110110001110001000000000000decoded   : { timeMs: 1791722121659, dc: 3, worker: 17, seq: 0 } -> 2026-10-11T12:35:21.659Zgenerated 2,000,000 ids in 512 ms (3,904,308 ids/s, one process, this machine). overflow waits: 473

That is 3.9 million IDs per second from one JavaScript process on this machine, which is close to the 4.096 million per second ceiling of the 12-bit counter. The 473 overflow waits are the counter hitting 4,096 in a millisecond and the loop waiting for the next millisecond. The ceiling is part of the design: a worker cannot run faster than 4,096 per ms, by construction.

Tests. I ran the 8 tests with node --test --test-reporter=spec snowflake.test.ts:

✔ fields round-trip through decode✔ same millisecond: sequence counts up, ids strictly increase✔ sequence overflow at 4096: waits for next millisecond✔ small clock step back: waits, then continues, never repeats or goes down✔ big clock step back: refuses to issue ids✔ two workers with different worker ids never collide, even in the same ms✔ ids fit in 63 bits (positive signed 64-bit) and bad config is rejected✔ real clock: 200,000 ids are unique and increasingℹ tests 8   ℹ pass 8   ℹ fail 0

The clock is injected (now and sleep are options), so the overflow and clock tests are exact and take milliseconds. A test that waits for the real clock to step back would be flaky. This is the most useful habit in the whole lesson: make time a parameter.

6.6 The three failures: clock, overflow, duplicate worker

1. The clock goes backwards. NTP can step a clock back, a virtual machine can pause and resume, a leap second can repeat. If the new time is before the last ID's time, naive code reuses an old millisecond and counter value, and you can issue a duplicate. Twitter's README says: "If your clock is running fast and NTP tells it to repeat a few milliseconds, snowflake will refuse to generate ids until a time that is after the last time we generated an id." My choices, tested above:

Better: run the time service in a mode that never steps back (it only slows down or speeds up the clock; the README hints at this, and chrony's "slew" mode is the usual tool, from memory). Another option is to keep lastMs in durable storage, so a restart after a long step back does not reuse old time.

2. Sequence overflow. More than 4,096 IDs in one millisecond on one worker. Options: wait for the next millisecond (what I do; costs under 1 ms), or throw. Never wrap the counter silently. Test 3 above proves the 4,097th ID goes into the next millisecond with sequence 0.

3. Two processes with the same worker number. This is the one that causes real duplicates, and no clock trick helps. It happens when a config file is copied, when a pod restarts and a lease is not released, or when two data centres are set up by hand. Fixes:

• Lease the worker number from a coordination store, with a time-to-live, and refuse to start without it.
• On crash, do not reuse the number until the lease has expired and the wall clock has passed the old lastMs (otherwise the new process could re-issue the same millisecond). A simple rule: wait one lease period before using a recycled number.
• Export the worker number as a metric, and alert on "two live processes claim worker 17".

6.7 Where Snowflake IDs hurt: JSON, sorting, and "k-sorted"

The JSON trap. JavaScript numbers lose precision above 2^53. I took a large 63-bit ID (a made-up value, not from the generator) and converted it:

== JSON / double trap ==bigint id        : 7381975040000012345as JS number     : 7381975040000012000 (round trip equal? false )

The last digits changed: ...012345 became ...012000. A browser calling JSON.parse on {"id": 7381975040000012345} would see the wrong ID, and your app would then fetch a different record (or none). Send IDs as strings in JSON, and in JavaScript parse them as BigInt or keep them as strings. (Twitter's API gives both id and id_str for this reason, from memory.)

Sorting is only approximate. IDs from two workers made in the same millisecond order by the worker number, not by real arrival order. With clock skew of a few ms between machines, IDs can be out of true order by that much. Twitter's README calls this k-sorted and promises "1s", "shooting for 10's of ms". So "sort by ID" is fine for a feed. It is not fine for "event A happened before B" in a ledger or an order book (Lesson 8); there you need one sequencer or a logical clock.

6.8 Ticket servers (Flickr-style) and block allocation

A ticket server is a database that is only a counter. Clients ask it for the next number. From memory, Flickr used two MySQL servers with odd and even offsets so that one could fail and the other keep serving. It is simple and gives dense, increasing numbers. The cost is a network round trip per ID and a pair of servers to operate.

The common fix is block allocation (also called a range or "hi/lo" allocator): the app asks for a whole block of 1,000 numbers, then hands them out from memory. I simulated it in ticket.ts:

interleaved ids from 2 app servers: 1, 1001, 2, 1002, 3, 1003, 4, 1004, 5, 1005A restarts, first new id: 2001 | ids 6..1000 are never used (a gap, no duplicate)1,000,000 ids, block size 1000 -> database round trips: 1000

One million IDs cost 1,000 database trips (a thousand times fewer). The price: when an app server crashes, the rest of its block is lost as a gap, and IDs from different servers are only roughly in order (1, 1001, 2, 1002...). Gaps are fine for IDs. They are not fine for "invoice numbers must be consecutive".

6.9 Choosing

My default for a new system: UUIDv7 if a 16-byte key is acceptable, Snowflake if I need 8 bytes and can run a lease service, plain auto-increment if it is one database and will stay that way.

7. Failure modes and trade-offs

Trade-offs to say aloud

• Ordered vs random: ordered IDs are index-friendly and leak time; random IDs hide time and hurt locality.
• 64 vs 128 bits: 8 bytes is smaller everywhere; 16 bytes needs no machine numbers.
• Coordination at startup vs per ID: a lease for worker numbers is one rare operation; asking a server for every ID is a bottleneck.
• Wait vs fail on clock problems: waiting hides a short issue, failing stops a long one. Pick a threshold and say it.
• Approximate vs exact order: k-sorted is cheap; exact order needs a single sequencer and a throughput limit.

8. How to say it in an interview (60 seconds)

"First I ask: how many IDs per second, do they need to sort by time, and is 64 bits needed. For one database I would just use auto-increment, but it fails with many writers and a 32-bit column fills in hours at 200,000 per second. UUIDv4 needs no coordination and is safe against collisions at about 2.7e18 IDs for a coin flip, but random keys hurt B-tree locality and take 16 bytes. UUIDv7 puts a 48-bit millisecond timestamp first and fixes that. If I need 64 bits I use a Snowflake layout: 41 bits of time is 69 years from a custom epoch, 10 bits of worker number gives 1,024 workers, 12 bits of sequence gives 4,096 IDs per millisecond per worker. Workers get their numbers by lease at startup, so the hot path has no network call. I handle three failures: if the clock goes back a few ms I wait, if more I refuse; if the sequence overflows I wait for the next millisecond; and I stop two processes sharing a worker number with leases. IDs go out as strings, because JavaScript loses precision above 2^53. And they are only roughly time-ordered, so I would not use them to decide event order in a ledger."

Run it yourself

Files: snowflake.ts (generator), snowflake.test.ts (8 tests), ids.ts (bit maths, UUID v4/v7, ULID, speed, JSON trap) and ticket.ts. In one folder run node --test snowflake.test.ts, node ids.ts and node ticket.ts. They use only Node's standard library and need Node 22.18 or newer (type stripping on by default; I used 22.22.0). Class parameter properties are avoided because type stripping does not support them.

Try it

Exercise 1. You change the layout to 39 bits time, 12 bits worker, 12 bits sequence (1 bit sign). How many years of IDs, how many workers, and how many IDs per second per worker? Answer: 2^39 ms = 549,755,813,888 ms = 17.4 years. Workers: 2^12 = 4,096. Per worker: 4,096 per ms = 4,096,000 per second. You traded 52 years of life for 4 times more workers. Check: 1 + 39 + 12 + 12 = 64.

Exercise 2. A worker issued an ID at millisecond 1000, sequence 4095. The clock then reads 1000 again. What must next() do? Answer: The sequence would wrap to 0 and repeat the pair (1000, 0). It must wait until the clock shows 1001 or later, then use sequence 0.

Exercise 3. Your IDs are BIGINT in the database and strings in the API. A teammate adds a front-end that does const id = response.id * 1. What breaks? Answer: Multiplying converts the string to a floating-point number and loses the low digits for IDs above 2^53 (about 9.0e15). Keep it a string or BigInt.

Interview questions

Key takeaways

• A 32-bit auto-increment column fills in hours at 200,000 IDs per second. A counter is fine for one database; many writers break it.
• UUIDv4 is safe against collisions (about 2.7e18 IDs for a coin flip) but random keys hurt B-tree locality. Random 64-bit IDs are not safe at scale (5e9).
• UUIDv7 and ULID put a 48-bit millisecond time first. I checked the UUIDv7 layout and test vector against the IETF working-group draft, not the final RFC text.
• Snowflake: 1 + 41 + 10 + 12 = 64 bits. 69.7 years of time, 1,024 workers, 4,096 IDs per ms per worker. Measured about 3.9 million IDs per second in one process.
• Three failures decide the design: clock going back (wait or refuse), sequence overflow (wait for the next ms), duplicate worker number (lease it).
• Send IDs as strings in JSON; a 63-bit number changes in a JavaScript double.
• IDs are roughly time-ordered, not exactly. Do not use them to decide event order where it matters.

What's next

Lesson 7: Design a System Across Multiple Data Centres. IDs that need no coordination are the first step. Next we look at everything else that must work when two data centres are far apart: replication lag, failover, split brain and data loss.