What happened

The cache fills from live queries until it hits a size cap, then evicts colder entries. With EDNS Client Subnet, the same query can produce multiple cached answers—so both entry count and bytes-per-entry matter.

Keys identify the query; values hold answer/authority/additional sections plus TTL and hit counters. Once stored, responses are immutable—but Vec/String still carried capacity fields and spare heap. Switching to Box<[T]> / Box<str> dropped eight bytes per field and over-allocation (~15 TB fleet-wide).

Separate section lists became one list with u16 offsets. Owners identical to the queried name are omitted and restored from the key. Large enum variants were boxed, then record payloads moved to contiguous wire bytes—better CPU-cache locality and less re-serialization on the hot path.

Why it matters

At “one byte × hundreds of billions,” struct padding is CapEx: Cloudflare equates 100 TB to about 130 Gen 13 servers. Speed improved with density, not despite it.

For non-DNS systems the pattern travels: measure dead capacity, alignment, and enum sizing against the heaviest variant. Pair synthetic benchmarks (traffic-like A/AAAA/TXT mix) with production RSS percentiles during staged rollout.

In practice

1. Prefer Box<[T]> when the buffer never grows after insert.
2. Merge sibling lists into one buffer with offsets when lifetimes match.
3. Audit enums: rare large variants inflate every small case.
4. Wire bytes on the read path can beat a fully parsed AST for common record types.
5. Ship one idea per release and watch p90/p99 resident memory—Cloudflare’s graph stepped down from May to July 2026.

Takeaway

Boring layout work bought terabytes and faster lookups. Freed memory is slated for more cache capacity—fewer upstream DNS queries.