Caching Strategies

Node.js Advanced — Caching Strategies

Node.js Advanced

Chapter 5 of 8 · Caching Strategies

Caching Strategies

A cache is a faster store that sits in front of a slower one. The slow store might be a database, an external API, or an expensive computation. Done well, caching cuts response times from hundreds of milliseconds to single-digit milliseconds and lets your database handle a fraction of the load. Done badly, it serves stale data nobody trusts, wastes memory on keys that never get read twice, or collapses under a thundering herd the moment the cache goes cold. This chapter covers the full picture: in-process caches, Redis patterns, cache-aside, invalidation strategies, and stampede prevention.

Why (and When) to Cache

Cache tier	Latency	Scope	Survives restart?	Best for
In-process Map/LRU	<1 µs	Single process only	No	Expensive computations, config, rarely-changing reference data
Redis	~0.5–2 ms	Shared across all processes/servers	Optional (AOF/RDB)	Session data, API responses, rate limiting, distributed locks
HTTP cache headers	0 ms (client-side)	Individual client / CDN	Yes (CDN)	Static assets, public API responses that don't vary per user

Don't cache everything. Cache only when: (1) the data is read far more than it's written, (2) the source is meaningfully slow, and (3) serving briefly stale data is acceptable or you have a reliable invalidation strategy.

In-Process Caching — TTL Map

// Simple TTL cache — no dependencies needed
class TtlCache {
    constructor(defaultTtlMs = 60_000) {
        this._store      = new Map();   // key → { value, expiresAt }
        this._defaultTtl = defaultTtlMs;
    }

    set(key, value, ttlMs = this._defaultTtl) {
        this._store.set(key, { value, expiresAt: Date.now() + ttlMs });
    }

    get(key) {
        const entry = this._store.get(key);
        if (!entry) return undefined;
        if (Date.now() > entry.expiresAt) {
            this._store.delete(key);   // lazy eviction
            return undefined;
        }
        return entry.value;
    }

    delete(key) { this._store.delete(key); }

    has(key)    { return this.get(key) !== undefined; }

    // Periodic sweep — call on an interval to reclaim memory from expired entries
    prune() {
        const now = Date.now();
        for (const [key, entry] of this._store) {
            if (now > entry.expiresAt) this._store.delete(key);
        }
    }
}

const cache = new TtlCache(30_000);   // 30-second default TTL
cache.set('config', { maxRetries: 3 });
cache.get('config');                  // { maxRetries: 3 }

const pruneTimer = setInterval(() => cache.prune(), 60_000);
pruneTimer.unref();                   // don't block process exit
    

LRU Cache — Bounded by Entry Count

A TTL cache evicts by age. An LRU (Least Recently Used) cache evicts the entry that was accessed furthest in the past when the cache hits its maximum size. The two are often combined: bounded by both count and TTL.

// npm install lru-cache  (the standard; used by npm itself)
const { LRUCache } = require('lru-cache');

const userCache = new LRUCache({
    max:  500,          // max 500 entries — oldest evicted when full
    ttl:  5 * 60_000,   // entries expire after 5 minutes
    // allowStale: true — return the stale value while fetching a fresh one
    // updateAgeOnGet: true — accessing a key resets its TTL
});

userCache.set('user:42', { id: 42, name: 'Alice' });
userCache.get('user:42');    // { id: 42, name: 'Alice' } — also resets its place in LRU order
userCache.delete('user:42');  // explicit invalidation

// fetchMethod: automatic cache-aside in one step (LRU cache v7+)
const autoCache = new LRUCache({
    max: 200,
    ttl: 60_000,
    fetchMethod: async (key) => {
        const id = key.replace('user:', '');
        return db.getUser(id);   // called only on cache miss
    },
});

const user = await autoCache.fetch('user:42');  // hit → instant; miss → DB query + cache
    

Cache-Aside Pattern (Lazy Loading)

Cache-aside is the most common pattern: the application code is responsible for populating and invalidating the cache. The cache never talks to the database directly.

Cache HIT Cache MISS Request Request │ │ ▼ ▼ Cache ──── key found ────► Return Cache ──── key not found │ ▼ Database │ ▼ Write to cache ◄── set TTL here │ ▼ Return

// Generic cache-aside helper
async function cacheAside(cache, key, fetcher, ttlMs) {
    const cached = cache.get(key);
    if (cached !== undefined) return cached;      // HIT

    const fresh = await fetcher();                // MISS — go to the source
    cache.set(key, fresh, ttlMs);
    return fresh;
}

// Usage
async function getUser(id) {
    return cacheAside(
        userCache,
        `user:${id}`,
        () => db.query('SELECT * FROM users WHERE id = ?', [id]),
        5 * 60_000,
    );
}

// Invalidate on write
async function updateUser(id, data) {
    await db.query('UPDATE users SET ? WHERE id = ?', [data, id]);
    userCache.delete(`user:${id}`);    // evict the stale entry
}
    

Redis — Shared Cache Across Processes

// npm install ioredis
const Redis = require('ioredis');

const redis = new Redis({
    host:           'localhost',
    port:           6379,
    maxRetriesPerRequest: 3,
    lazyConnect:    true,          // don't connect until first command
    enableReadyCheck: true,
});

redis.on('error', (err) => console.error('Redis error:', err.message));

// Basic string ops
await redis.set('key', 'value');                  // no expiry
await redis.set('key', 'value', 'EX', 300);       // expire in 300 seconds
await redis.setex('key', 300, 'value');           // shorthand for above
await redis.get('key');                             // 'value' or null
await redis.del('key');                             // delete
await redis.exists('key');                          // 1 or 0
await redis.ttl('key');                             // seconds remaining (-2 if gone)

// JSON values — Redis only stores strings, so serialize/deserialize
await redis.set('user:42', JSON.stringify({ id: 42, name: 'Alice' }), 'EX', 300);
const raw  = await redis.get('user:42');
const user = raw ? JSON.parse(raw) : null;
    

Redis Cache-Aside (Distributed)

async function getUserCached(id) {
    const key = `user:${id}`;

    const cached = await redis.get(key);
    if (cached) return JSON.parse(cached);       // HIT

    const user = await db.getUser(id);           // MISS
    if (user) {
        await redis.set(key, JSON.stringify(user), 'EX', 300);
    }
    return user;
}

// Batch reads — pipeline avoids N round-trips
async function getUsersBatch(ids) {
    const keys    = ids.map(id => `user:${id}`);
    const results = await redis.mget(...keys);       // single round-trip for all keys

    const misses  = [];
    const users   = results.map((raw, i) => {
        if (raw) return JSON.parse(raw);
        misses.push(ids[i]);
        return null;
    });

    if (misses.length > 0) {
        const fresh = await db.getUsersByIds(misses);
        const pipe  = redis.pipeline();
        for (const u of fresh) {
            pipe.set(`user:${u.id}`, JSON.stringify(u), 'EX', 300);
        }
        await pipe.exec();                          // flush all SET commands in one round-trip

        const freshMap = Object.fromEntries(fresh.map(u => [u.id, u]));
        users.forEach((u, i) => { if (!u) users[i] = freshMap[ids[i]] ?? null; });
    }

    return users;
}
    

Cache Invalidation Strategies

TTL-Based (Time to Live)

Every key expires after N seconds. The simplest strategy — no explicit invalidation code needed. Trade-off: data can be stale for up to TTL seconds after a write.

Tune TTL to your staleness tolerance. EX 300 for user profiles; EX 5 for live inventory counts.

Event-Based (Active Invalidation)

When a record is written, explicitly delete its cache key: redis.del(`user:${id}`). The next read repopulates from the DB.

Stronger consistency than TTL alone — the window between write and stale eviction drops to near-zero. Requires discipline: every write path must also invalidate.

Cache Versioning (Namespace Bumping)

Store a version number: user:v3:42. When the schema or logic changes, bump the version prefix and all old keys become orphaned (and expire naturally via TTL).

Useful for mass invalidations without a FLUSHDB — just bump the global version.

Tag-Based Invalidation

Tag each cached value with logical groups (tag:product:99). When product 99 changes, delete every key in its tag set. Redis Sets work well for this: SADD tag:product:99 "key1" "key2" → on update, SMEMBERS + DEL all members.

Cache Stampede (Thundering Herd)

A cache stampede happens when a popular key expires and dozens of requests all miss simultaneously — they all query the database in parallel, pounding it with identical queries. The fix is to ensure only one request populates the cache while others wait or get a slightly stale value.

STAMPEDE — key expires, 50 requests all miss at once t=0 key expires t=1 Request 1 ──► Cache MISS ──► DB query ─────────────────► write cache t=1 Request 2 ──► Cache MISS ──► DB query (duplicate!) ──────► t=1 Request 3 ──► Cache MISS ──► DB query (duplicate!) ──────► ...50 simultaneous DB queries for the same row... FIX — distributed lock: only one request queries, others wait or return stale t=0 key expires t=1 Request 1 ──► Miss ──► SETNX lock:key ──► DB query ──► write cache ──► release lock t=1 Request 2 ──► Miss ──► SETNX lock:key FAILS ──► wait / return stale t=1 Request 3 ──► Miss ──► SETNX lock:key FAILS ──► wait / return stale

// Stampede prevention with a Redis distributed lock
async function getWithLock(key, fetcher, ttlSec = 300) {
    // 1. Try cache first (fast path)
    const cached = await redis.get(key);
    if (cached) return JSON.parse(cached);

    const lockKey = `lock:${key}`;

    // 2. Try to acquire the lock (SET NX = only if not exists; EX = auto-release)
    const acquired = await redis.set(lockKey, '1', 'NX', 'EX', 10);

    if (acquired) {
        // We hold the lock — fetch and populate
        try {
            const fresh = await fetcher();
            await redis.set(key, JSON.stringify(fresh), 'EX', ttlSec);
            return fresh;
        } finally {
            await redis.del(lockKey);   // always release
        }
    }

    // 3. Another request holds the lock — poll briefly then retry
    await new Promise(r => setTimeout(r, 50));
    const retried = await redis.get(key);
    return retried ? JSON.parse(retried) : getWithLock(key, fetcher, ttlSec);
}
    

// Probabilistic Early Expiry — simpler alternative to locking
// Randomly refresh the cache slightly before it expires, so one request
// recomputes proactively rather than all requests missing at the exact TTL.
async function getWithEarlyExpiry(key, fetcher, ttlSec, beta = 1) {
    const raw = await redis.get(key);

    if (raw) {
        const { value, storedAt, computedInMs } = JSON.parse(raw);
        const remainingSec = (storedAt + ttlSec * 1000 - Date.now()) / 1000;
        const earlyScore   = -beta * (computedInMs / 1000) * Math.log(Math.random());

        if (remainingSec > earlyScore) return value;   // still fresh enough
        // else: fall through and recompute (probabilistic early refresh)
    }

    const start = Date.now();
    const fresh = await fetcher();
    const payload = { value: fresh, storedAt: Date.now(), computedInMs: Date.now() - start };
    await redis.set(key, JSON.stringify(payload), 'EX', ttlSec);
    return fresh;
}
    

HTTP Cache Headers

// Cache-Control for public API responses (e.g. product catalogue)
res.set('Cache-Control', 'public, max-age=300, stale-while-revalidate=60');
// max-age=300       → CDN/browser can cache for 5 minutes
// stale-while-revalidate=60 → serve stale for up to 60s while fetching fresh

// ETag-based conditional requests (avoids transferring unchanged bodies)
const crypto = require('node:crypto');

app.get('/products', async (req, res) => {
    const products = await getProductsCached();
    const etag     = `"${crypto.createHash('md5').update(JSON.stringify(products)).digest('hex')}"`;

    res.set('ETag', etag);

    if (req.headers['if-none-match'] === etag) {
        return res.status(304).end();   // Not Modified — body transfer skipped
    }

    res.json(products);
});

// Private user data — never cache at CDN level
res.set('Cache-Control', 'private, no-store');
    

⚠️ Cache Key Design — Get This Wrong and You Cache the Wrong User's Data

Cache keys must include every variable that affects the result. A route that returns different data per user must include the user ID in the key: user:42:orders not just orders. A route that varies by language must include locale: products:en:42.

Never use raw user input directly as a cache key — sanitise it first. A key like `search:${userInput}` can be poisoned with special characters, or explode the key space with unique garbage values that never get a second hit.

💡 Key Naming Convention

Use entity:id:field format: user:42, user:42:orders, product:99:stock. Colons are Redis's conventional namespace separator — they don't have special meaning to Redis but make redis-cli --scan --pattern "user:*" queries readable.

Prefix with your app name in shared Redis instances: myapp:user:42. Avoid key collisions between services.

Coding Challenges

Challenge 1 — LRU Cache from Scratch

Implement an LRUCache class backed by a doubly-linked list and a Map (no external packages). It must support: get(key) (O(1), moves accessed node to head), set(key, value, ttlMs) (O(1), evicts LRU tail when over capacity, respects per-entry TTL), delete(key) (O(1)), and a read-only size property. Write tests proving: LRU eviction order is correct, TTL expiry removes entries, and all operations are O(1).

View sample solution ↗

Challenge 2 — Redis Cache Wrapper with Stats

Build a RedisCache class wrapping ioredis that adds: getOrSet(key, fetcher, ttlSec) implementing cache-aside, a hit/miss counter accessible via stats() returning { hits, misses, hitRate }, and invalidate(...keys) that deletes multiple keys in a single pipeline call. Write tests using ioredis-mock (or a hand-rolled mock) that verify: a cold get calls the fetcher exactly once, a warm get skips the fetcher, stats accumulate correctly, and invalidate clears the right keys.

View sample solution ↗

Challenge 3 — Stampede-Proof Cache

Build stampedeSafeGet(redis, key, fetcher, ttlSec) that prevents thundering herd using a Redis lock (SET NX EX). Prove it works by simulating 20 concurrent callers all requesting the same cold key simultaneously — assert that the underlying fetcher is called exactly once no matter how many concurrent requests arrive. Use ioredis-mock for Redis and control timing with jest.useFakeTimers() or a manual promise barrier. Also handle the edge case where the lock holder crashes (lock auto-expires) so waiters don't block forever.

View sample solution ↗