← All posts

layercache: stopping cache stampedes in Node.js

L1 + Redis, single-flight, and graceful degradation — layercache benchmarks against thundering herds.

Tags
BackendNewsNode.js
Contents

In brief

The classic Node.js path: in-memory Map → Redis when you scale → a hybrid that breaks at 2 AM. On Dev.to, the author benchmarked layercache — a layered cache with cache stampede protection, Redis failure modes, and cross-instance coordination. HTTP benchmarks showed up to ~100× throughput versus an uncached route.

What happened

layercache stacks tiers: L1 in-process memory (0.01 ms), L2 Redis (0.5 ms), optional L3 on disk. Lower hits backfill upper layers — an L2 hit repopulates L1 automatically.

The painful case is the thundering herd: a key expires and 75 concurrent requests hammer the database 75 times. Single-flight via a coordination lock in Redis (or memory) leaves one fetcher; others wait. In a 375-request cold-key test, the origin ran once per expiry round, not 75 times.

On Express + autocannon (40 connections, 8 s): /nocache ~161 req/s, /memory and /layered ~17k req/s with p97 around 4 ms — memory-only and layered look alike on the hot path, while Redis backs L2 across pods.

With 500 ms injected Redis latency, hot L1 keys stayed fast; cold misses against dead Redis still hit timeout (~2 s) — graceful degradation protects warmed keys, not brand-new ones.

At maxSize: 25 with 180 × 256 KB payloads, L1 evicted old entries but revisits loaded from L2, not the origin. Two Node.js instances sharing Redis: Pub/Sub invalidation and 60 parallel requests for one missing key → a single origin fetch.

Why it matters

Hand-rolled caches rarely handle stampedes and multi-replica consistency. Without coordination, horizontal scale multiplies database load whenever a popular TTL expires.

layercache does not remove ops discipline: warm critical keys before Redis incidents, and size L1 generously for large payloads — 1 MB serialization in Redis was ~200× slower than memory hits in the benchmarks.

In practice

  1. For multiple instances, use a layered stack (L1 + Redis) with single-flight, not a local Map alone.
  2. Warm hot keys before traffic spikes; cold misses on slow Redis can cost seconds.
  3. Do not expect graceful degradation for new keys when Redis is fully down — only warmed L1 helps.
  4. Set L1 maxSize with headroom for large objects.
  5. Wire RedisInvalidationBus and RedisSingleFlightCoordinator for clusters.
const cache = new CacheStack([
  new MemoryLayer({ ttl: 60, maxSize: 10_000 }),
  new RedisLayer({ client: redis, ttl: 3600 })
])
const user = await cache.get('user:123', () => db.findUser(123))

Takeaway

layercache addresses stampede prevention, distributed invalidation, and partial degradation without sacrificing L1 speed on the hot path. For services beyond a single process, it beats another homemade ioredis wrapper.