Concurrency: Promises, Worker Threads, Clusters and Child Processes

In lesson 6 we made a server stop politely. Now we make it do more work. There are two very different kinds of "slow", and beginners mix them up:

1. Waiting. Calling a database, an API, or the disk. The CPU is idle while you wait. Node is excellent at this, and one thread can have thousands of waits in progress.
2. Calculating. Hashing, resizing images, parsing a giant file, computing Fibonacci. The CPU is busy. While your one JavaScript thread is calculating, it cannot do anything else. (If you want the full picture of why, read lesson 1, the event loop.)

For waiting, the tools are Promise.all, Promise.allSettled, timeouts and concurrency limits. For calculating, the tools are worker_threads, cluster and child_process. We will time each one with real numbers.

If you want more on blocking the loop, see Backend Lesson 10 (Node.js performance basics).

Words used in this lesson

• Concurrency means several tasks are in progress at the same time (they may take turns).
• Parallelism means several tasks are running at the same time on different CPU cores.
• A promise is an object that stands for a value that will arrive later (or an error).
• Promise.all starts waiting for many promises and gives you all results, or fails as soon as one fails.
• A concurrency limit is a cap on how many tasks may run at the same moment.
• Rate limiting (by the server) means the other side rejects you if you send too much too fast (often with a 429 or 503).
• A thread is a path of execution inside one process. Threads of one process can share memory.
• A process is a running program with its own memory. Processes do not share memory by default.
• A worker thread is a second JavaScript thread inside your Node process, with its own event loop and its own copy of the JS engine state.
• Structured clone is the copying method used to send a JavaScript value to another thread.
• A transferable is a value (like an ArrayBuffer) that is moved to another thread instead of copied.
• SharedArrayBuffer is a block of memory that several threads can read and write together.
• An atomic operation is one that cannot be interrupted halfway by another thread.
• Cluster is a Node module that starts several copies (worker processes) of your server that share one port.
• A child process is another program (or another Node script) started from your program.
• A thread pool (worker pool) is a fixed set of workers that are created once and reused.

Promise.all and friends: waiting at the same time

Let us fake a slow call. getUser waits 300 ms and returns a user. We need five users.

import { setTimeout as sleep } from "node:timers/promises";const getUser = async (id) => { await sleep(300); return { id, name: "user" + id }; };

One after another with await inside a loop:

const out = [];for (const id of [1, 2, 3, 4, 5]) out.push(await getUser(id));

All at once with Promise.all:

const users = await Promise.all([1, 2, 3, 4, 5].map(getUser));

one after another      1507 msPromise.all             301 ms

Five waits of 300 ms took 1.5 seconds in a row and 0.3 seconds together. The reason is simple: the waits do not use the CPU, so Node can have all five open at once. This is the most common speed-up in Node code: independent awaits should not be in a sequence.

Four ways to combine promises

The behaviour on failure is what differs. I used a function that waits 100 * id ms and fails for id === 2:

Promise.all        -> REJECTED: user 2 failed   (after 201 ms)Promise.allSettled -> [ 'fulfilled', 'rejected', 'fulfilled' ]   (after 301 ms)Promise.race       -> "fast"    (the first one to finish, success or failure, after 51 ms)Promise.any        -> 3         (the first one to succeed; here id 3; after 301 ms)

Timeouts with AbortSignal

A call that never finishes is a classic production outage. AbortSignal.timeout(ms) makes a signal that fires after ms milliseconds. Many Node APIs, including fetch and timers/promises, accept it. I pointed fetch at a server that never answers:

try {  await fetch("http://localhost:4101", { signal: AbortSignal.timeout(400) });} catch (e) {  console.log("fetch:", e.name);}

fetch: TimeoutError 404 ms

The request stopped after about 400 ms and threw a TimeoutError that we can handle. Set a timeout on every network call you make. A missing timeout is one of the most common causes of "the whole server is stuck".

The forEach trap

forEach does not wait for async callbacks:

const out = [];[1, 2, 3].forEach(async (n) => { await sleep(10); out.push(n); });console.log("after forEach:", out);await sleep(50);console.log("later:", out);

after forEach: []later: [ 1, 2, 3 ]

The line after forEach ran before any callback finished. Use for...of with await for a sequence, or await Promise.all(items.map(async ...)) for parallel work.

Too much concurrency: why you need a limit

Promise.all on a list of 5 is fine. On a list of 5000 it may start 5000 requests at once. Every real service has a limit: a database pool of 10 connections, an API allowing 5 requests at the same time, a file-descriptor limit on your own machine. Let us make that limit real. Here is a tiny API (on localhost) that can serve only 5 requests at once. Above that it returns 503. Each request takes 200 ms.

let active = 0, peak = 0;const server = http.createServer(async (req, res) => {  active++; peak = Math.max(peak, active);  if (active > 5) { active--; res.statusCode = 503; return res.end("busy"); }  await new Promise(r => setTimeout(r, 200));  active--; res.end("ok");});

Now I need 30 items. First the naive way, everything at once:

const r = await Promise.all(ids.map(call));

A small concurrency limiter

Here is the fix in about 12 lines. We start a fixed number of "workers" (they are just async loops). Each one takes the next item from a shared counter until the list is empty.

async function mapLimit(items, limit, fn) {  const results = new Array(items.length);  let next = 0;  async function worker() {    while (next < items.length) {      const i = next++;                 // safe: JS is single-threaded, no await between read and ++      results[i] = await fn(items[i], i);    }  }  await Promise.all(Array.from({ length: limit }, worker));  return results;}

How it works: limit copies of worker() start. Each loops: take an index, await the job, take the next index. At any moment at most limit jobs are running. The comment is important: next++ is safe without a lock because JavaScript never switches tasks in the middle of a synchronous line. (With real threads, as you will see below, that would not be safe.)

The results:

Promise.all, 30 at once : { '200': 5, '503': 25 } peak on server: 6   254 msmapLimit(5)             : { '200': 30 }          peak on server: 5   1226 msmapLimit(1) (sequential): { '200': 30 }          peak on server: 1   6078 ms

Read these three lines carefully:

• All at once: very fast (254 ms) but 25 out of 30 failed with 503. The "speed" is fake, because most of the work did not happen.
• Limit of 5: all 30 succeeded in 1.2 seconds. The server never saw more than 5 at a time. (30 jobs / 5 at a time = 6 rounds x 200 ms = 1.2 s.)
• Limit of 1: all succeeded but it took 6 seconds.

The right number is the largest concurrency the other side can handle, usually found by measuring. In practice, many teams use a package such as p-limit or p-queue instead of writing their own. I wrote the small version so the idea is clear. I did not run those packages in this lesson.

For a server handling many incoming requests, the same idea appears as a database connection pool: a fixed number of connections shared by all requests, with extra requests waiting in line.

CPU work blocks everything

Now the second kind of slow. Here is a function that burns CPU: a slow recursive Fibonacci.

function fib(n) { return n < 2 ? n : fib(n - 1) + fib(n - 2); }

fib(40) takes a bit over a second on my machine. I compute it 4 times on the main thread. At the same time, a "heartbeat" timer wants to run every 100 ms. I measure how late the heartbeat was:

let last = performance.now(), worst = 0;setInterval(() => { const now = performance.now(); worst = Math.max(worst, now - last - 100); last = now; }, 100);const results = [40, 40, 40, 40].map(fib);   // 4 jobs, one after another, on the main thread

--- main threadresults [ 102334155, 102334155, 102334155, 102334155 ] total 5115 msworst loop delay: 5026 ms

Total 5.1 seconds, and the heartbeat that should have fired every 100 ms was late by 5 seconds. For all that time the event loop was frozen: no timers, no incoming requests, no responses. In a server this means every user waits.

To see it in a real server, I made an HTTP server with /fib (computes fib(40)) and /health (returns "ok"). I started a /fib request, and 0.2 seconds later asked for /health:

== main thread versionhealth took 1.075691s

A health check that should take 3 milliseconds took more than a second, just because someone else's request was calculating. If your load balancer's health check has a 1 second timeout, it would decide your server is dead.

Which Node work does not block?

Some work looks heavy but Node runs it off the main thread already. File system calls, DNS lookups, and many crypto functions (like pbkdf2, scrypt, randomBytes) run in libuv's thread pool (4 threads by default). Network I/O uses the operating system's async features. That is why a Node server can handle thousands of connections. JavaScript code that you write, however, always runs on the main thread. The Node docs on worker_threads say the same thing from the other side: workers help with CPU-heavy JavaScript and do not help much with I/O, since the built-in async I/O is already more efficient.

worker_threads: parallel JavaScript

A worker thread is a second JavaScript thread in the same process, with its own event loop. You send it a message with the input, it computes, and it sends a message back. Neither side blocks the other.

The worker file:

// fib-worker.mjsimport { parentPort, workerData } from "node:worker_threads";function fib(n) { return n < 2 ? n : fib(n - 1) + fib(n - 2); }parentPort.postMessage(fib(workerData));

The main file:

import { Worker } from "node:worker_threads";const run = (n) => new Promise((resolve, reject) => {  const w = new Worker(new URL("./fib-worker.mjs", import.meta.url), { workerData: n });  w.once("message", resolve);  w.once("error", reject);});const results = await Promise.all([40, 40, 40, 40].map(run));   // 4 threads at once

--- 4 workersresults [ 102334155, 102334155, 102334155, 102334155 ] total 2255 msworst loop delay: 6 ms

Compare with the main-thread run:

Two things improved. The total time dropped by about 2.3 times (not 4 times, even though I have 4 cores: my sandbox shares CPUs, and starting threads has a cost). And, more important for a server, the event loop stayed free: the worst delay was 6 ms instead of 5 seconds.

The server test confirms it:

== worker versionhealth took 0.003014s

/health answered in 3 ms while a fib(40) request was running in a worker.

Workers are not free

Creating a worker takes time and memory (it starts a new JavaScript engine instance). To measure it I ran 40 small tasks (fib(25), about a millisecond each) three ways:

A) 40 tasks, new Worker each time  : 695 msB) 40 tasks, pool of 4 workers     : 61 ms     (a second run: 115 ms)C) 40 tasks on the main thread     : 36 ms

This is an honest and slightly surprising result. For tiny tasks:

• Starting a new worker for each task (A) was the slowest by far, around 17 ms of overhead per worker.
• A pool of 4 reused workers (B) was much better, but still not faster than just doing the work on the main thread (C).

The rule: use workers for tasks that take long enough to matter (tens of milliseconds and up), and reuse them with a pool. A pool is simple. Create N workers once, keep a queue of jobs, and give each job to an idle worker. My pool class in the test was about 20 lines. In real projects people often use a package for this (for example piscina), but I did not install or run it here, so I cannot show output for it.

class Pool {  constructor(size) {    this.idle = []; this.queue = []; this.jobs = new Map(); this.nextId = 0;    for (let i = 0; i < size; i++) {      const w = new Worker(new URL("./pool-worker.mjs", import.meta.url));      w.on("message", ({ id, result }) => {        this.jobs.get(id)(result); this.jobs.delete(id);        this.idle.push(w); this.pump();      });      this.idle.push(w);    }  }  run(n) { return new Promise((resolve) => { this.queue.push({ n, resolve }); this.pump(); }); }  pump() {    while (this.idle.length && this.queue.length) {      const w = this.idle.pop(), { n, resolve } = this.queue.shift(), id = this.nextId++;      this.jobs.set(id, resolve); w.postMessage({ id, n });    }  }}

This sketch has no error handling (a worker that crashes would leave its job hanging). A real pool must handle the 'error' and 'exit' events and replace dead workers.

Passing data: copy, transfer, share

By default postMessage copies the data with the structured clone algorithm. For big data this is slow. A 200 MB Uint8Array:

const big = new Uint8Array(200 * 1024 * 1024);structuredClone(big);                                   // copystructuredClone(big, { transfer: [big.buffer] });       // move

clone    200MB: 2612 ms, original length 209715200transfer 200MB: 0.51 ms, original length now 0

(The 2.6 seconds is high partly because my sandbox is slow with big allocations. Look at the ratio.) A transfer moves ownership: it is instant, but the original array becomes empty (length 0). That is the price: only one side owns the memory at a time. structuredClone uses the same algorithm as postMessage, so the numbers apply to messages.

The third option is shared memory: a SharedArrayBuffer that both threads see. It is the fastest, and the most dangerous, because now two threads can write at the same time. A test with 4 workers each adding 1 to a shared counter 100,000 times:

const counter = new Int32Array(new SharedArrayBuffer(4));// worker, plain version:   counter[0]++// worker, atomic version:  Atomics.add(counter, 0, 1)

plain   expected 400000, got 393894atomic  expected 400000, got 400000

With plain counter[0]++, updates were lost (about 6,000 missing in this run; the number changes each time). That is a race condition: two threads read the same value, both add one, both write back, and one update vanishes. Atomics.add does the read-add-write as one uninterruptible step and gives the right answer. This problem does not exist in normal single-threaded JavaScript, which is why the limiter code earlier did not need a lock.

cluster: several copies of the whole server

worker_threads helps when one request has heavy CPU work. Another problem is different: one Node process uses at most one core for JavaScript, so a 4-core machine is 75% idle. The cluster module starts several worker processes of your server that share one port.

import cluster from "node:cluster";import http from "node:http";if (cluster.isPrimary) {  console.log("primary", process.pid, "starting 3 workers");  for (let i = 0; i < 3; i++) cluster.fork();  cluster.on("exit", (w, code) => console.log("worker", w.process.pid, "exited", code));} else {  http.createServer((req, res) => res.end("handled by " + process.pid + "\n")).listen(4300);}

The same file runs several times: once as the primary, and once for each fork (where isPrimary is false). I sent 6 requests with curl and counted which process answered:

      2 handled by 5527      2 handled by 5528      2 handled by 5534

Three different process ids, two requests each. The Node cluster docs describe the default on Linux and macOS as round-robin: the primary accepts connections and hands them out in turn. That matches the even split I saw.

Pros and cons:

Cluster today: often replaced by containers

In many modern setups you do not write cluster code. You run one Node process per container and let the platform start N containers (Kubernetes replicas, ECS tasks, several instances behind a load balancer). The load balancer does the round-robin, the platform restarts crashed containers, and you can scale across machines, not just cores. Cluster still makes sense on a single server (for example a VM without an orchestrator), and a process manager like PM2 offers a cluster mode. For the infrastructure side, see DevOps Lesson 9 (Docker).

Whatever you choose, remember the shutdown lesson: each worker or container must handle SIGTERM and drain.

child_process: run another program

child_process starts a separate program. It can be any program (git, ffmpeg, python), or another Node script. The main functions:

I ran each:

const { stdout } = await execFileP("node", ["-e", "console.log(process.version)"]);console.log("execFile:", stdout.trim());

execFile: v22.22.0

spawn streams output while the program runs, and tells you the exit code at the end. I ran a small shell script that prints one, waits, prints two, writes oops to stderr, and exits with code 3:

spawn stdout: onespawn stdout: twospawn stderr: oopsspawn exit code: 3

Errors work like this: a non-zero exit makes execFile (the promise version) reject with e.code set to the exit code, and a missing program rejects with ENOENT:

execFile failed, code: 2missing program: ENOENT

fork also gives a message channel, like a worker thread but in a separate process:

// child: process.on("message", m => process.send({ got: m, pid: process.pid }));const c = fork("./fk-child.mjs");c.on("message", (m) => { console.log("parent", process.pid, "got", m); c.kill(); });c.send("ping");

parent 5583 got { got: 'ping', pid: 5590 }

The injection trap: exec versus execFile

This is a security point you must know. exec builds a shell command from a string. If part of that string comes from a user, the user can add shell syntax. I used the name x; echo INJECTED:

exec(`echo hello ${name}`, ...);execFile("echo", ["hello", name], ...);

exec    -> "hello x\nINJECTED\n"execFile-> "hello x; echo INJECTED\n"

With exec, the shell saw ; as "end of command, start a new one", and ran echo INJECTED. An attacker could use ; rm -rf ... instead. With execFile, the whole text is just one argument to echo, so it was printed as plain text. The Node docs say clearly to never pass unsanitised user input to exec. Rule: use execFile or spawn with an arguments array, and avoid the shell. See Security Lesson 1 (injection) for the full story.

Choosing the right tool

The last row matters. Often the best design is not a worker thread inside your web server but a separate process that takes jobs from a queue. The web request answers "accepted" right away, and the job runs elsewhere. That also lets you scale the heavy part separately.

Try it

1. Write mapLimit yourself. Test it with 20 fake tasks of 100 ms each and a limit of 4. Measure the total time and the peak number of running tasks.
2. Make Promise.all fail on purpose with one rejected promise, while the other promises keep logging "still running". What do you see?
3. Run fib(38) on the main thread inside a server route, and request /health while it runs. Then move it into a worker. Measure /health in both cases with curl -w "%{time_total}".
4. Run the shared counter test with and without Atomics.add. Run it five times. Is the wrong number always the same?
5. Compare exec and execFile with the input a && echo HACKED.

Answers

1. 20 tasks / 4 at a time = 5 rounds x 100 ms = about 500 ms. Peak is 4 (count it with an active variable).
2. Promise.all rejects early, but the other promises still finish and log. A rejection does not cancel work.
3. Main thread: /health waits for the whole calculation (about a second or more). Worker: /health answers in a few milliseconds.
4. Without atomics the result is below the expected value and changes from run to run. With Atomics.add it is always exact.
5. exec prints both a and HACKED. execFile("echo", ["a && echo HACKED"]) prints the text once, as plain text.

Interview questions

Key takeaways

• Waiting and calculating are different problems. Node handles waiting well on one thread. Calculating blocks the loop.
• Independent awaits belong in Promise.all. In my test: 1507 ms in sequence against 301 ms together.
• Promise.all rejects early but does not cancel the others. Use AbortSignal for real cancellation and set a timeout on every network call.
• Unlimited concurrency breaks fragile services: 25 of 30 requests failed. A small mapLimit fixed it with a measured peak of 5.
• Heavy JavaScript blocks everyone: a fib(40) request made /health take over 1 second. In a worker, /health took 3 ms.
• Workers cost time to create. Reuse them in a pool, and only for work that is not tiny.
• Copying big data is slow, transferring is instant but empties the source, shared memory needs Atomics.
• cluster runs many processes on one port. In container setups, one process per container usually replaces it.
• Use execFile/spawn with an argument array. exec with user input is an injection hole.

What's next

Next: Testing, Performance and Production Habits. We will write tests with the built-in node:test, profile a slow script with --cpu-prof, hunt a memory leak with real heap numbers, and finish with configuration, Docker and a production checklist. Practise now: take any for loop with await inside from your own code and ask whether it can run with a limit instead.