How much linear memory access is enough? New benchmarks say “not as much as you think”

It has been reported that a new set of microbenchmarks from Philip Trettner digs into a deceptively simple question: when you must process data in contiguous blocks, how big do those blocks actually need to be before you stop losing performance by jumping between them? The headline takeaway is blunt and useful: 1 MB blocks are enough for basically any workload of this kind; 128 kB suffices once you do roughly one cycle of work per byte; and 4 kB is often enough when you’re doing about ten cycles per byte. Surprising? A little. Relieving? Absolutely.

How the experiment worked

Trettner built a tightly controlled kernel that walks spans of contiguous float blocks and computes simple running statistics, returning a small hash so the optimizer can’t toss the work. The setup varies the block size while keeping the total working set constant, randomizes block placement to avoid warm-cache illusions, and reports median timings on a Ryzen 9 7950X3D. It has been reported that the author targeted realistic per-element work rates to show where the amortized cost of a block jump disappears, and the full code and raw data are available on GitHub (github.com/solidean/bench-linear-access) for anyone who wants to poke or reproduce the results.

Why this matters — and where to be cautious

The practical lesson: you don’t need to obsess over global contiguity for every data layout decision. Want to break a 32 GB dataset into smaller, more flexible blocks? Fine. If your inner loop is cheap, err toward larger blocks; if your per-item work is heavy, the block size becomes much less important. Caveats remain — it has been reported that these numbers apply to raw-data processing and may shift when per-block setup costs, I/O, or different CPU architectures enter the picture — so don’t treat the figures as gospel for every platform. But for systems programmers and library authors wrestling with cache hierarchies and 3D V‑Cache machines, this is a welcome, empirically grounded nudge: the contiguity fetish can be dialed back.

Sources: solidean.com, Lobsters