Worker Threads

Node.js Intermediate — Worker Threads

Node.js Intermediate

Chapter 4 of 8 · Worker Threads

Worker Threads

Child processes (Chapter 3) isolate heavy work by spawning a whole new OS process — safe, but expensive (~30 MB per process, slow startup, no shared memory). Worker Threads give you true multi-threading inside a single Node process: lower overhead, fast startup, and the ability to share memory via SharedArrayBuffer. The event loop in each thread remains single-threaded, but multiple threads can run JavaScript in parallel.

The worker_threads Module

The four key exports you'll use in almost every worker setup:

isMainThread

true in the main thread, false in any worker. Lets a single file serve as both the orchestrator and the worker.

parentPort

The MessagePort a worker uses to communicate with its parent. null in the main thread. Call .postMessage() and listen with .on('message').

workerData

A structured-clone of the data passed as { workerData: ... } when creating the worker. Read-only initial configuration — available immediately, before any messages.

Worker

The class that spawns a new thread from a file or inline code string. Gives you .postMessage(), on('message'), on('error'), and on('exit') on the main-thread side.

Basic Worker Setup — Single File Pattern

The cleanest approach is a single file that checks isMainThread to decide whether it's the orchestrator or the worker.

const { Worker, isMainThread, parentPort, workerData } =
    require('worker_threads');

if (isMainThread) {
    // ── Main thread ─────────────────────────────────
    const worker = new Worker(__filename, {
        workerData: { start: 1, end: 10_000_000 }
    });

    worker.on('message', (result) => {
        console.log('Sum:', result);  // arrives without blocking the event loop
    });

    worker.on('error', (err) => console.error('Worker error:', err));

    worker.on('exit', (code) => {
        if (code !== 0) console.error(`Worker exited with code ${code}`);
    });

    console.log('Main thread is free while the worker runs...');

} else {
    // ── Worker thread ────────────────────────────────
    const { start, end } = workerData;
    let sum = 0;
    for (let i = start; i <= end; i++) sum += i;
    parentPort.postMessage(sum);  // send result back; thread exits naturally
}
    

💡 Worker Script as a Separate File

If you prefer to keep the worker code in its own file, pass the path to the Worker constructor instead of __filename:

new Worker('./worker.js', { workerData: { ... } })

There's no functional difference — use whichever makes the module boundaries cleaner for your project.

Bidirectional Messaging

Workers and the main thread can send messages in both directions at any time — it's not a strict request/response model. Messages are structured-cloned (deep copy), so there's no shared state risk with plain objects.

// Main thread — sends tasks, receives progress + final result
const worker = new Worker(__filename, { workerData: { batchSize: 1000 } });

worker.postMessage({ type: 'start', dataset: largeArray });

worker.on('message', (msg) => {
    if (msg.type === 'progress') {
        console.log(`Progress: ${msg.percent}%`);
    } else if (msg.type === 'done') {
        console.log('Finished:', msg.result);
        worker.terminate();
    }
});
    

// Worker thread — receives a task, reports progress, sends final result
const { batchSize } = workerData;

parentPort.on('message', ({ type, dataset }) => {
    if (type !== 'start') return;

    let result = 0;
    for (let i = 0; i < dataset.length; i++) {
        result += dataset[i];

        if (i % batchSize === 0) {
            const percent = Math.round((i / dataset.length) * 100);
            parentPort.postMessage({ type: 'progress', percent });
        }
    }

    parentPort.postMessage({ type: 'done', result });
});
    

SharedArrayBuffer — Zero-Copy Shared Memory

Structured cloning copies data on every postMessage. For large datasets that multiple workers need to read simultaneously, SharedArrayBuffer lets all threads read from the same memory without copying.

// Main thread — creates a shared buffer, hands it to the worker
const sharedBuffer = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT * 4);
const sharedArray  = new Int32Array(sharedBuffer);

sharedArray[0] = 10;  // write before handing off
sharedArray[1] = 20;

const worker = new Worker(__filename, {
    workerData: { sharedBuffer }   // SAB is passed by reference, not copied
});

worker.on('message', () => {
    console.log('Worker wrote:', sharedArray[2]);  // reads what the worker set
});
    

// Worker thread — reads from and writes to the same memory
const { sharedBuffer } = workerData;
const arr = new Int32Array(sharedBuffer);

console.log('Worker sees:', arr[0], arr[1]);  // 10, 20
arr[2] = arr[0] + arr[1];                     // writes 30 directly into shared memory
parentPort.postMessage('done');
    

⚠️ Race Conditions with SharedArrayBuffer

Two threads writing to the same index simultaneously produces unpredictable results. If multiple workers need to mutate shared data concurrently, use Atomics to make individual operations thread-safe.

Atomics — Thread-Safe Operations

Atomics provides operations that complete without interruption, even when multiple threads are running simultaneously. They only work on integer TypedArray views over a SharedArrayBuffer.

const buf     = new SharedArrayBuffer(4);
const counter = new Int32Array(buf);

// Atomic add — returns the OLD value, increments atomically
const prev = Atomics.add(counter, 0, 1);

// Compare-and-swap — only writes if the current value matches expected
const succeeded = Atomics.compareExchange(counter, 0, prev + 1, 99);
// index=0, expectedValue=prev+1, replacementValue=99
// Returns the actual value found — compare to expected to know if swap succeeded

// Load / store — atomic read and write
const val = Atomics.load(counter, 0);
Atomics.store(counter, 0, 42);

// Wait / notify — for synchronisation between threads
// Atomics.wait()   blocks until the value at index changes (worker threads only)
// Atomics.notify() wakes up threads waiting at an index
Atomics.notify(counter, 0, 1);  // wake up 1 thread waiting at index 0
    

Transferable Objects — Zero-Copy Message Passing

For large ArrayBuffers (not SharedArrayBuffer), you can transfer ownership instead of copying. The sender loses access to the buffer — only the receiver can use it after the transfer.

const buffer = new ArrayBuffer(1024 * 1024 * 100);  // 100 MB

// Transfer (move ownership — zero copy, buffer is now detached in sender)
worker.postMessage({ buffer }, [buffer]);  // second arg = transferList

console.log(buffer.byteLength);  // 0 — sender can no longer use it

MessageChannel — Direct Worker-to-Worker Communication

By default, workers can only talk to the main thread. MessageChannel creates a connected port pair that can be transferred to different threads, enabling workers to communicate directly.

const { MessageChannel } = require('worker_threads');

const { port1, port2 } = new MessageChannel();

// Transfer port1 to worker A, port2 to worker B
workerA.postMessage({ port: port1 }, [port1]);
workerB.postMessage({ port: port2 }, [port2]);

// Workers can now send directly to each other:
// In worker A:  port.postMessage({ hello: 'from A' })
// In worker B:  port.on('message', (msg) => console.log(msg))
    

Building a Worker Pool

Creating a new Worker for each task wastes startup time. A pool keeps N workers alive and recycles them across tasks — the same pattern as the TaskFarm from Chapter 3, but with threads instead of processes.

const { Worker, isMainThread, parentPort, workerData } =
    require('worker_threads');

if (!isMainThread) {
    // Worker: process tasks as they arrive, stay alive between them
    parentPort.on('message', ({ taskId, payload }) => {
        const result = heavyCompute(payload);
        parentPort.postMessage({ taskId, result });
    });
    return;
}

// ── Main thread — Worker Pool ────────────────────────────────────
class WorkerPool {
    constructor(size = 4) {
        this._pending  = new Map();   // taskId → { resolve, reject }
        this._taskId   = 0;
        this._next     = 0;
        this._workers  = Array.from({ length: size }, () => {
            const w = new Worker(__filename);
            w.on('message', ({ taskId, result }) => {
                const { resolve } = this._pending.get(taskId);
                this._pending.delete(taskId);
                resolve(result);
            });
            w.on('error', (err) => console.error('Worker error:', err));
            return w;
        });
    }

    run(payload) {
        return new Promise((resolve, reject) => {
            const taskId = this._taskId++;
            this._pending.set(taskId, { resolve, reject });
            const worker = this._workers[this._next++ % this._workers.length];
            worker.postMessage({ taskId, payload });
        });
    }

    async shutdown() {
        for (const w of this._workers) await w.terminate();
    }
}

const pool = new WorkerPool(4);
const results = await Promise.all(
    [40, 41, 42, 43].map(n => pool.run({ n }))
);
console.log(results);
await pool.shutdown();
    

Worker Pool — round-robin dispatch Main Thread │ ├─ pool.run(task1) ──► Worker 0 ── heavyCompute() ──► postMessage(result) ├─ pool.run(task2) ──► Worker 1 ── heavyCompute() ──► postMessage(result) ├─ pool.run(task3) ──► Worker 2 ── heavyCompute() ──► postMessage(result) └─ pool.run(task4) ──► Worker 3 ── heavyCompute() ──► postMessage(result) All four tasks run in parallel — main thread stays free

Worker Threads vs Child Processes vs async I/O

Approach	Memory	Startup	Best for
async I/O	Shared (same thread)	None	Waiting on network, files, databases — the event loop handles concurrency automatically
Worker Threads	Shared (`SharedArrayBuffer`) or copied	~5 ms	CPU-intensive JavaScript — image processing, compression, parsing, crypto — in the same process
Child Processes	Separate (copied or piped)	~30 ms + 30 MB	Running shell tools, other executables, or Node scripts that need full process isolation

💡 Most Node.js Apps Don't Need Worker Threads

If your bottleneck is waiting on a database, an API, or a file — that's I/O, and the event loop already handles it without threads. Worker Threads only help when JavaScript itself is doing expensive synchronous computation that blocks the loop (e.g., resizing images, computing hashes over large payloads, parsing huge JSON). Profile first; threads add complexity.

⚠️ What Workers Can't Do

Access the DOM — there is no DOM in Node, but browser workers similarly can't touch the UI thread's DOM.

Share non-serialisable objects — functions, class instances, Promises, and Buffers backed by Node internals can't cross thread boundaries via postMessage. Use SharedArrayBuffer for large binary data or plain-object messages for everything else.

Call Atomics.wait() from the main thread — it would block the event loop. Use Atomics.waitAsync() (non-blocking) from the main thread instead.

Coding Challenges

Challenge 1 — Parallel Array Processing

Write a parallelMap(array, workerFile, concurrency) function that splits an array into concurrency equal chunks, sends each chunk to a separate Worker Thread, and resolves with the combined results in the original order. The worker file receives { chunk } via workerData and posts back the processed chunk. Test it by squaring an array of 1 million numbers across 4 threads.

View sample solution ↗

Challenge 2 — Shared Counter with Atomics

Create a benchmark that spawns 4 workers, each incrementing a shared counter 250,000 times (for a total of 1,000,000 increments). Run it twice: once using plain sharedArray[0]++ (unsafe) and once using Atomics.add(sharedArray, 0, 1) (safe). Log the final counter value in both cases and explain why the unsafe version is likely to produce a number less than 1,000,000.

View sample solution ↗

Challenge 3 — Progress-Reporting Worker Pool

Extend the WorkerPool from the chapter so that workers can emit progress events mid-task. The pool.run(payload, onProgress) method should accept an optional callback that fires whenever the worker posts { type: 'progress', taskId, percent }. Final results still resolve the returned Promise via { type: 'done', taskId, result }. Demonstrate it by running a simulated long task that reports progress at 25%, 50%, 75%, and 100%.

View sample solution ↗