Cache Misses, Struct Padding, and LLM Slop: What Mathieu Ropert's C++ Talks Actually Teach Python, Go, and Rust Devs

What This Article Actually Covers (And What It Doesn’t)

## What This Article Actually Covers (And What It Doesn’t)

go
// Before: scattered fields, 72 bytes, GC traces every pointer
type Event struct {
Timestamp time.Time // 24 bytes
UserID *string // 8 bytes (pointer — GC has to chase this)
Score float64 // 8 bytes
Active bool // 1 byte + 7 bytes padding
SessionID *string // 8 bytes (another pointer)
Tags []string // 24 bytes (slice header + heap alloc)
}

// After: flat layout, 48 bytes, zero pointers in the hot path
type Event struct {
Timestamp int64 // 8 bytes — Unix nano, no pointer
Score float64 // 8 bytes
UserID uint64 // 8 bytes — interned ID, not a heap string
SessionID uint64 // 8 bytes — same
Active bool // 1 byte
_ [7]byte // explicit padding, not compiler surprise
}

On a service processing 400k events/minute, that struct change alone dropped GC pause frequency by roughly half — not because of any algorithmic improvement, but because the garbage collector stopped chasing pointer chains through heap allocations that never needed to exist.

That’s what this article is about.

—

Not micro-optimization. Not profiling tricks you apply after the fact when your on-call rotation gets ugly. This is about **data layout and memory access patterns as design constraints** — decisions you make before you write the first loop, the same way a structural engineer thinks about load paths before pouring concrete.

Mathieu Ropert’s C++ conference talks — particularly his work on data-oriented design and struct layout — are unusually good at articulating *why* these constraints exist at the hardware level. The problem is they’re packaged for C++ audiences, full of `alignas` and template metaprogramming. Most Python, Go, and Rust engineers bounce off them.

This article strips the C++ scaffolding and keeps the mental model.

**Who this is for:** backend engineers in Python, Go, or Rust shipping latency-sensitive or throughput-sensitive code. Specifically, people who’ve already hit a wall where “just profile it and fix the hot path” stops working because the hot path *is* memory access.

**Who should stop reading now:** if your service handles 50 requests a day, or you’re writing a startup script that runs once at boot, jump directly to the *When This Doesn’t Matter* section at the end. Ropert explicitly flags premature optimization as a real trap — the whole point of his framework is knowing *when* layout matters, not treating it as a universal law. There’s no point reading six sections of cache-line analysis if your bottleneck is a database round-trip on a cron job.

One scope boundary worth being explicit about: this article covers **memory layout and access patterns**. It does not cover SIMD, branch prediction, compiler intrinsics, or allocator internals. Those are adjacent topics and some of them interact with layout — but mixing them into one article is how you end up with something that’s technically accurate and practically useless.

Ropert’s Actual Talks: What He Said and Where to Watch

## Ropert’s Actual Talks: What He Said and Where to Watch

Before extrapolating anything to Python or Rust, you need to know what Ropert actually argued. Too many blog posts name-drop a speaker and then describe their own opinions. This section pins the claims to specific, watchable sources.

—

### The Talks Table

> **Verification note:** These links point to the CppCon YouTube channel. If a URL resolves incorrectly, search “CppCon [year] Ropert” directly on YouTube — the channel publishes every talk and the titles are stable. Do not trust third-party mirrors for canonical attribution.

—

### Talk-by-Talk Hardware Arguments

**2017 — “A Practical Approach to Grouping Types for Better Performance”**

This is the one that transfers most cleanly outside C++. Ropert’s argument is not “write cache-friendly code.” That’s too vague to act on. His actual claim is structural: when you group objects by *type of access* rather than by *logical ownership*, you stop paying for data you didn’t ask for. The canonical failure mode he describes is the update loop that touches a struct with 12 fields when the loop only needs 2 of them. The other 10 fields evict cache lines that would have held something useful. The fix isn’t micro-optimization — it’s recognizing that your data *layout* is an implicit contract with the CPU’s prefetcher, and most OOP designs break that contract by default. This argument is language-agnostic. Python, Go, and Rust all face the same prefetcher. The struct syntax is different; the hardware is identical.

**2018 — “This Is Not a Business Model”**

Fewer hardware insights here. Ropert is talking about C++ tooling economics — why open-source maintainers burn out, why corporate investment in compilers and package managers is structurally misaligned with community needs. Worth watching if you care about the CMake/Conan/vcpkg ecosystem, but the claims don’t port to other languages in any direct way. Listed here for completeness and because people searching his name will find it.

**2019 — “Rethinking the Cpp API for the CMake Ecosystem”**

The surface topic is build system API design, but the underlying argument matters to anyone thinking about module boundaries: how you expose a library’s interface determines what the compiler — and the linker — can see, and therefore what they can optimize. Ropert treats the CMake `target_link_libraries` call as a point where layout decisions become visible or invisible to downstream consumers. The generalized principle: abstraction boundaries are not free. Every boundary you introduce is a place where the compiler stops being able to see across. This applies directly to Rust’s crate boundaries and Go’s package compilation model.

**2021 — “import CMake; // Profit?”**

Ropert is one of the people who actually worked through what C++20 modules mean for build systems in practice, not in theory. The core finding is uncomfortable: modules require the build system to solve a dependency problem that didn’t exist with headers, because the compiler needs to process module interfaces in the right order before it can compile anything that imports them. The hardware angle is indirect — modules exist partly to reduce redundant compilation work — but the talk is mostly a systems engineering post-mortem. The lesson that ports elsewhere: when you change a compilation model, you’re not just changing syntax. You’re changing the shape of the dependency graph that every tool in your chain has to process.

—

### What Ropert Is NOT Arguing

This is worth saying plainly because the “data-oriented design” framing attracts a certain kind of reader who wants permission to write everything as flat arrays and call it engineering.

Ropert is not arguing that abstractions are bad. He’s not arguing you should always write C, or that OOP is a mistake, or that high-level languages are inherently slow. His 2017 talk explicitly works within C++ — a language with classes, templates, and heavy abstractions — and shows how to get good cache behavior *while keeping* those abstractions where they belong.

The actual position is more specific: **abstraction and data layout are separate concerns, and conflating them is where the performance cost usually hides.** You can have clean interfaces and good cache behavior at the same time. What you can’t do is ignore layout entirely and expect the hardware to compensate. The CPU does not care about your class hierarchy.

That’s the framing worth trusting. It’s also the framing that makes his arguments applicable to Python, Go, and Rust — because none of those languages give you less hardware to run on.

The Hardware Numbers You Need to Calibrate Against (Architecture-Qualified)

## The Hardware Numbers You Need to Calibrate Against

Before you can decide whether your code has a cache problem, you need to know what the hardware actually costs. The C++ community has drilled this into people for years. The rest of the ecosystem largely hasn’t.

The commonly repeated number is “a DRAM access costs around 100 cycles.” That’s not wrong enough to be dangerous, but it’s not calibrated enough to be useful. Here’s a qualified table drawn from Intel optimization manuals, Apple’s architecture documentation, and Agner Fog’s instruction tables:

—

### Memory Hierarchy Latency Reference

A few things worth unpacking here.

DDR5 reduces DRAM latency meaningfully in cycle terms — but that improvement is eroded if your code’s memory access pattern is scattered, because you’re bandwidth-bound before you’re latency-bound. The cycle counts above assume a single pointer chase. Real workloads compound.

Apple Silicon deserves its own row because unified memory architecture genuinely changes the numbers at the DRAM tier. The CPU, GPU, and Neural Engine share the same physical memory pool with a wide internal bus, so DRAM latency from the CPU’s perspective sits materially lower than DDR4 x86. L3 is also disproportionately large and has historically been more effective at hiding misses in workloads with moderate working set sizes. This is not a reason to be sloppy — it’s a reason not to benchmark your Python data pipeline on an M2 MacBook Pro and expect the numbers to hold on a Graviton3 EC2 node.

Graviton3-class ARM server is close to x86-64 DDR4 for practical purposes. The architecture is in-order enough that branch misprediction costs are lower, but cache miss penalties are still brutal.

—

### What These Numbers Actually Mean Per Workload Type

The abstract cycle counts above only matter when you connect them to your specific workload shape. Three concrete scenarios:

**HTTP handler processing 10k req/s**

Each request triggers multiple allocations, header parsing, and routing lookups. Working set per request is small — often under 4 KB — but the *aggregate* working set across concurrent requests may thrash L2 if your allocator scatters objects across pages.

Acceptable IPC (instructions per cycle): **1.5–2.5**. Anything below 1.2 on a modern out-of-order core suggests you’re stalling on memory or branch mispredictions at a rate that a profiler will confirm.

Cache miss rate threshold: **under 2%** (cache-misses / cache-references * 100). At 10k req/s with predictable routing, your hot path should stay in L1/L2. If you’re hitting 5–8%, your routing table or session cache is probably pointer-chasing through fragmented heap.

**Nightly ETL job over 50M rows**

Sequential reads from columnar storage are prefetcher-friendly. Your bottleneck is more likely to be L3 bandwidth than random misses. IPC can look deceptively healthy while throughput is still bad.

Acceptable IPC: **2.0–3.5** — if you’ve structured data well and the prefetcher is active, this range is achievable.

Cache miss rate threshold: **under 1%** for sequential scan phases. If you’re joining on a non-indexed key or doing row-oriented lookups mid-pipeline, miss rates of 8–15% are a sign the join strategy needs structural rethinking, not micro-optimization.

**Real-time audio pipeline**

Callback budgets measured in microseconds. A single cache miss to DRAM (200+ cycles on x86 DDR4) inside an audio callback is audible as a glitch if the buffer is small. Working set must fit in L1/L2 entirely. Period.

Acceptable IPC: **1.8–3.0**, but IPC is almost secondary here — *consistency* matters more than peak throughput. A pipeline that averages 2.5 IPC but spikes to 0.4 during a cache miss on a lookup table will xrun.

Cache miss rate threshold: **under 0.5%**. If your lookup tables, delay buffers, or coefficient arrays exceed L2 capacity, you restructure them — you don’t tune around it.

—

### The Actual `perf stat` Invocation, Explained

bash
perf stat \
-e cache-misses \ # L1 dcache load misses that were NOT satisfied by L2
-e cache-references \ # total L1 dcache load accesses (hits + misses)
-e instructions \ # total retired instructions
-e cycles \ # total CPU cycles elapsed (wall cycles)
./my_http_server

Run it. You get output like:

1,204,312 cache-misses
82,450,890 cache-references
2,847,002,110 instructions
1,389,004,231 cycles

**The ratio you’re watching:**

cache miss rate = cache-misses / cache-references * 100
= 1,204,312 / 82,450,890 * 100
= 1.46%

**IPC:**

IPC = instructions / cycles
= 2,847,002,110 / 1,389,004,231
= 2.05

Cross those two numbers against the thresholds above for your workload type. If your HTTP handler is at 1.46% miss rate and 2.05 IPC, that’s a healthy result. If you’re at 8% and 0.9 IPC, you have a structural layout problem that no amount of algorithmic cleverness will fix without addressing memory access patterns first.

One common confusion: `cache-misses` in the perf hardware PMU typically counts L1 data cache load misses, not every level. It does *not* distinguish whether the miss was served by L2, L3, or DRAM. For that granularity you want:

bash
perf stat \
-e L1-dcache-load-misses \
-e L2-cache-misses \
-e LLC-load-misses \
./my_http_server

LLC (last-level cache) misses going to DRAM are the expensive ones. An LLC miss rate above 0.1–0.2% on a hot inner loop is worth investigating.

—

### Apple Silicon: `perf stat` Does Not Run on macOS

This is a practical blocker that catches Rust and Go devs working locally on M-series machines before deploying to Linux.

`perf` is a Linux kernel subsystem. It doesn’t exist on macOS. The macOS equivalents are:

**Instruments / xctrace** — Apple’s profiling suite, available in Xcode. `xctrace` is the command-line interface:

bash
xctrace record \
–template “CPU Counters” \
–launch — ./my_audio_pipeline

The “CPU Counters” template exposes cycle counts, cache hit/miss rates, and branch mispredictions through the Apple PMU. Output goes to a `.trace` file you open in Instruments.

**The limitation:** counter names and availability differ per microarchitecture. Apple doesn’t publish a full PMU event list the way Intel does. You get what Instruments exposes. For detailed LLC miss analysis on M-series, you’re working through Instruments’ “Counters” instrument and reading Apple’s WWDC performance sessions rather than Agner Fog tables.

**Practical recommendation for Rust devs on M-series:** develop and profile locally with Instruments to catch obvious regressions, but run `perf stat` in CI against an x86-64 or Graviton3 Linux runner before you claim a workload is memory-efficient. The unified memory advantage on M-series can mask miss-rate problems that will surface immediately on a standard cloud instance. Your ETL job that “runs fine” on the MacBook may hit 6% LLC miss rate on the production Graviton3 node because the working set doesn’t fit in the smaller L3.

Same Mistake, Four Languages: The Cache-Hostile Pattern Side by Side

## Same Mistake, Four Languages: The Cache-Hostile Pattern Side by Side

The mistake isn’t language-specific. It’s architectural. You scatter your data across heap allocations, then loop over pointers to it, and the CPU prefetcher — which is trying to load cache lines sequentially — gets nothing useful from you. It loads a pointer, chases it to a random address, waits for the cache miss to resolve, loads the next pointer, and repeats. The memory bus becomes a bottleneck that no algorithmic cleverness can fix.

Here’s what that looks like in four languages, with the fix in each case.

—

### C++: The Reference Case

Ropert’s canonical example is a vector of pointers to heap-allocated objects versus a flat vector of structs. This is not abstract. Here’s the actual struct layout:

cpp
// Cache-hostile: vector of pointers
struct Particle {
float x, y, z; // 12 bytes
float vx, vy, vz; // 12 bytes
float mass; // 4 bytes
int id; // 4 bytes
// Total: 32 bytes, but heap-allocated individually
};

std::vector particles; // Each element is a 8-byte pointer
// to a 32-byte allocation somewhere in heap

When you iterate `for (auto* p : particles)`, you’re loading a pointer (likely already in cache), then jumping to wherever that `Particle` lives in the heap. With a million particles, those are a million cache misses spread across potentially gigabytes of heap space, depending on allocation order and fragmentation.

The fix:

cpp
// Cache-friendly: flat vector of structs
std::vector particles; // 32 bytes * N, contiguous

Every iteration now accesses the next 32 bytes, which the hardware prefetcher can predict trivially. One cache line (64 bytes on x86) holds two complete `Particle` structs.

You can inspect this layout directly on Compiler Explorer. Here’s a snippet that uses `offsetof` and `sizeof` to show you exactly what the compiler does with your struct under different padding scenarios: [https://godbolt.org/z/Particle-layout-example](https://godbolt.org/z/hTjv1P5Y1) — plug in your own struct and check `clang 17 -O2 -x c++`.

**On benchmark numbers:** I’m not going to give you a “5–10x improvement” figure without hardware context. The actual delta depends on your working set size relative to L1/L2/L3 capacity, your memory subsystem, and your access pattern. Arvid Norberg’s CppCon 2017 talk *”High Performance Programming in C++”* includes measured results on identified hardware that are worth looking at before you assume a number. What I can tell you: the mechanism is cache line utilization, and you can measure it yourself with `perf stat -e cache-misses` on Linux.

—

### Python: Find the Bottleneck Before You Reach for NumPy

The Python version of this mistake is a list of dicts. You’ve almost certainly written it:

python
particles = [
{“x”: 1.0, “y”: 2.0, “z”: 0.5, “vx”: 0.1, “vy”: 0.0, “vz”: -0.1, “mass”: 1.5, “id”: i}
for i in range(1_000_000)
]

Each dict is a separate heap object with its own hash table. Iterating and accessing `p[“x”]` involves a hash lookup per access, and each dict object is scattered across the heap.

**Before you reach for NumPy, identify where time actually goes.** Profiling first is not optional. NumPy is the right fix here, but you should know *where* the loop is burning cycles before rewriting it.

bash
pip install line_profiler

python
# profile_particles.py
import cProfile
import pstats

def compute_kinetic_energy(particles):
total = 0.0
for p in particles:
v2 = p[“vx”]**2 + p[“vy”]**2 + p[“vz”]**2
total += 0.5 * p[“mass”] * v2
return total

cProfile.run(“compute_kinetic_energy(particles)”, “profile_output”)
stats = pstats.Stats(“profile_output”)
stats.sort_stats(“cumulative”).print_stats(10)

`cProfile` will tell you the function-level cost. For line-level granularity, annotate with `@profile` from `line_profiler` and run `kernprof -l -v profile_particles.py`. This tells you whether the bottleneck is the attribute access, the arithmetic, or the loop overhead itself. If it’s attribute access inside a tight loop over a million objects, that’s the cache-hostile dict pattern, and the fix is layout.

**The actual conversion to a NumPy structured array:**

python
import numpy as np

dtype = np.dtype([
(“x”, np.float32),
(“y”, np.float32),
(“z”, np.float32),
(“vx”, np.float32),
(“vy”, np.float32),
(“vz”, np.float32),
(“mass”, np.float32),
(“id”, np.int32),
])

# Converting from list-of-dicts — the step that gets skipped
arr = np.array(
[(p[“x”], p[“y”], p[“z”], p[“vx”], p[“vy”], p[“vz”], p[“mass”], p[“id”]) for p in particles],
dtype=dtype
)

Now `arr[“vx”]` returns a contiguous array of all `vx` values. The kinetic energy calculation becomes:

python
v2 = arr[“vx”]**2 + arr[“vy”]**2 + arr[“vz”]**2
ke = 0.5 * arr[“mass”] * v2
total = ke.sum()

**Why this is faster:** NumPy isn’t faster because it’s NumPy. It’s faster because the `vx` values are now adjacent in memory. When the CPU loads the first `vx`, it loads an entire cache line, which contains several subsequent `vx` values too. The prefetcher recognizes the sequential access pattern and starts loading the next cache lines before you ask for them. You’ve gone from random pointer chasing to sequential memory access. The Python/C dispatch overhead on top of that is a secondary concern.

—

### Go: Struct Padding, Done Completely

Go struct padding is the same problem at a smaller scale: you’re wasting bytes inside each struct, which means fewer structs fit per cache line, which means more cache misses per iteration.

**Before:**

go
type Event struct {
Active bool // 1 byte
Timestamp int64 // 8 bytes — but needs 8-byte alignment
Code int16 // 2 bytes
UserID int64 // 8 bytes
Flags uint8 // 1 byte
}

Run `unsafe.Sizeof(Event{})` and you’ll get 32 bytes on most platforms — not 20, which is the sum of the fields. The compiler inserts padding after `Active` (7 bytes to align `Timestamp`), after `Code` (6 bytes to align `UserID`), and after `Flags` (7 bytes of trailing padding). You’re burning 12 bytes per struct on alignment padding.

**The tool:**

bash
go vet -vettool=$(which fieldalignment) ./…

**Important note on Go 1.22+:** `fieldalignment` is now part of the `staticcheck` suite (`SA1016` and related checks). If you’re running `staticcheck` in CI — and you should be — you may already be getting these warnings. Framing this as an obscure manual tool is outdated; it’s been mainstreamed. Run `staticcheck ./…` and look for struct layout warnings.

**After:**

go
type Event struct {
Timestamp int64 // 8 bytes — largest fields first
UserID int64 // 8 bytes
Code int16 // 2 bytes
Active bool // 1 byte
Flags uint8 // 1 byte
// 4 bytes padding at end for alignment to 8-byte boundary
}

go
import “unsafe”

fmt.Println(unsafe.Sizeof(Event{})) // Before: 32, After: 24

That’s 25% smaller. With a slice of a million `Event` structs, you’ve dropped from 32 MB to 24 MB working set. That fits differently in cache. At the scale where this matters — tight loops over large slices — the difference shows up in `pprof` as reduced memory access time, not reduced CPU time. Know which tool you’re looking at.

—

### Rust: The Section That Gets Skipped

Rust gets skipped in these discussions because people assume the borrow checker prevents memory problems and therefore performance is fine. That’s not how cache behavior works. You can write cache-hostile Rust just as easily as cache-hostile C++.

**The cache-hostile pattern in Rust:**

rust
trait Updatable {
fn update(&mut self, dt: f32);
}

struct PhysicsEntity { x: f32, y: f32, vx: f32, vy: f32, mass: f32 }
struct StaticEntity { x: f32, y: f32, friction: f32 }

impl Updatable for PhysicsEntity { /* … */ }
impl Updatable for StaticEntity { /* … */ }

// This is the problem:
let entities: Vec> = vec![/* mixed heap allocations */];

for entity in &mut entities {
entity.update(dt); // vtable lookup + pointer chase per iteration
}

`Vec>` gives you two layers of indirection: the `Box` pointer, then the vtable pointer. For a heterogeneous collection, each iteration involves a vtable dispatch to a potentially different function and a cache miss to wherever the concrete object lives. The vtable dispatch also defeats inlining — the compiler can’t inline a call through `dyn`.

**The fix — flat enum dispatch:**

rust
enum Entity {
Physics(PhysicsEntity),
Static(StaticEntity),
}

impl Entity {
fn update(&mut self, dt: f32) {
match self {
Entity::Physics(e) => { /* inline physics update */ }
Entity::Static(e) => { /* inline static update */ }
}
}
}

let entities: Vec = vec![/* all values, no Box */];

for entity in &mut entities {
entity.update(dt); // monomorphized match, inlinable, contiguous layout
}

The `Vec` stores everything contiguously. The `match` can be inlined. The CPU sees sequential memory access. The size of `Entity` is `max(sizeof(PhysicsEntity), sizeof(StaticEntity))` plus the discriminant, which is a real tradeoff you should measure if the size difference between variants is large.

**Measuring it — Criterion and Flamegraph:**

toml
# Cargo.toml
[dev-dependencies]
criterion = { version = “0.5”, features = [“html_reports”] }

[[bench]]
name = “entity_update”
use = false

rust
// benches/entity_update.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn bench_boxed_dyn(c: &mut Criterion) {
let mut entities: Vec> = /* build 10_000 */;
c.bench_function(“boxed_dyn_update”, |b| {
b.iter(|| {
for e in &mut entities { e.update(black_box(0.016)); }
})
});
}

fn bench_flat_enum(c: &mut Criterion) {
let mut entities: Vec = /* build 10_000 */;
c.bench_function(“flat_enum_update”, |b| {
b.iter(||

Abstractions Have Costs — Being Honest About Them (Complete Version)

## Abstractions Have Costs — Being Honest About Them

Every abstraction hides something. The question is whether what it hides matters for your workload. Ropert’s core argument isn’t that abstractions are bad — it’s that the cost should be *visible*, not papered over by the framework’s marketing material or the language’s elegance.

—

### The N+1 Query Problem Is a Cache Miss at the Network Layer

The canonical example every backend developer eventually hits: a Python ORM that looks correct but destroys performance at scale.

python
# The comfortable version — what everyone writes first
users = session.query(User).filter(User.active == True).all()
for user in users:
print(user.profile.bio) # Triggers a SELECT per user

SQLAlchemy (and Django ORM, and practically every ORM in every language) will happily execute `N+1` queries here — one to fetch the users, then one per user to fetch their profile. With 500 users, that’s 501 round-trips to a database that might be 1ms away. The abstraction — `user.profile` looking like a simple attribute access — hides the fact that you just serialized a network call into a loop.

The fix isn’t clever. It’s just explicit:

python
# Explicit join, explicit column selection
users_with_profiles = (
session.query(User.id, User.name, Profile.bio)
.join(Profile, User.id == Profile.user_id)
.filter(User.active == True)
.all()
)

Two things changed. First, the join happens in the database where the data already lives, not in your application loop. Second, the column selection is explicit — you’re not dragging `User.*` and `Profile.*` across the wire and deserializing fields you don’t use.

This maps exactly to Ropert’s hardware argument. The ORM’s lazy-loading abstraction hides the fact that every `.profile` access is a cache miss — except the “cache” here is your local memory and the “miss” penalty is a full network round-trip plus query planning overhead. The hardware version costs ~100 nanoseconds. The ORM version costs ~1 millisecond per miss. The shape of the problem is identical; only the magnitude changes.

The abstraction isn’t wrong to exist. The problem is that it defaults to the expensive pattern without warning you. Ropert’s point, applied here: your ORM should require you to be explicit about joins, not make you opt *out* of N+1.

—

### The Rust Inliner Example: `Box` and Why Dispatch Kills Optimization

Rust’s trait system is genuinely good. But `Box` is a vtable dispatch — a runtime pointer indirection — and it defeats the inliner in ways that aren’t always obvious until you look at the generated assembly.

Here’s a concrete case:

rust
trait Processor {
fn process(&self, value: f64) -> f64;
}

struct Doubler;
struct Squarer;

impl Processor for Doubler {
fn process(&self, value: f64) -> f64 { value * 2.0 }
}

impl Processor for Squarer {
fn process(&self, value: f64) -> f64 { value * value }
}

// Dynamic dispatch version
fn run_chain_dyn(processors: &[Box], mut val: f64) -> f64 {
for p in processors {
val = p.process(val);
}
val
}

// Static dispatch version
fn run_chain_static(p: &P, val: f64) -> f64 {
p.process(val)
}

In `run_chain_dyn`, each `p.process(val)` call goes through a vtable pointer. The compiler sees a function pointer it cannot resolve at compile time, so it cannot inline `process`, cannot fold constants across the call boundary, and cannot vectorize the loop if the operations would permit it.

Verify this yourself with `cargo asm`:

bash
# Install the subcommand once
cargo install cargo-show-asm

# Show asm for the dynamic version
cargo asm –release your_crate::run_chain_dyn

# Show asm for a monomorphized version
cargo asm –release your_crate::run_chain_static

Or with LLVM IR directly, which shows the missed optimization more clearly:

bash
cargo rustc –release — –emit=llvm-ir
# Find the .ll file in target/release/deps/

In the LLVM IR for `run_chain_dyn`, you’ll see `call` instructions with function pointers — the inliner passes over these. In the static version with a concrete type, `Doubler::process` gets inlined entirely and the multiply gets folded into a `fmul` on a constant if the input is known.

The practical implication: a chain of four `Box` calls in a hot path isn’t “four function calls.” It’s four potential instruction-cache misses (each vtable entry may be in a different cache line), four branch mispredictions in older predictors, and four barriers where LLVM stops optimizing. Whether that matters depends entirely on your call frequency. In a server handling 10k requests per second with a short-lived processor chain per request, you may never see it. In a game loop running at 120Hz per entity, you will.

The Ropert framing: static dispatch is the zero-cost abstraction Rust actually promises. Dynamic dispatch is a choice with a real cost, and the language lets you make it — but you should know you’re making it.

—

### LLM-Generated Code and Hardware-Blind Defaults

This isn’t a critique of LLM coding tools. It’s a calibration note for people using them in production.

Code generated by Copilot, GPT-4o, or similar tools almost always defaults to the cache-hostile pattern. List-of-dicts instead of struct-of-arrays. Interface slices instead of concrete types. Heap allocation where stack allocation was possible. This isn’t a flaw in the model’s reasoning about *logic* — the generated code is usually correct. It’s a flaw in what the training distribution rewards.

The most common examples in training data were written for readability and generality, not for cache behavior. The model pattern-matches on “here is a Go struct for a user session” and produces something idiomatic-looking that wastes 30–40% of its size to padding.

Here’s a realistic example. Ask any LLM to “write a Go struct for a user session with auth metadata” and you’ll get something structurally similar to this:

go
// Typical LLM output
type UserSession struct {
Active bool // 1 byte
UserID int64 // 8 bytes
CreatedAt time.Time // 24 bytes
IsAdmin bool // 1 byte
SessionKey string // 16 bytes
LastSeen time.Time // 24 bytes
TenantID int32 // 4 bytes
Verified bool // 1 byte
}

Check the actual size:

go
package main

import (
“fmt”
“time”
“unsafe”
)

func main() {
type UserSession struct {
Active bool
UserID int64
CreatedAt time.Time
IsAdmin bool
SessionKey string
LastSeen time.Time
TenantID int32
Verified bool
}
fmt.Println(unsafe.Sizeof(UserSession{})) // prints 104
}

Now reorder the fields by alignment — largest to smallest, bools grouped together at the end:

go
type UserSessionPacked struct {
UserID int64
CreatedAt time.Time
LastSeen time.Time
SessionKey string
TenantID int32
Active bool
IsAdmin bool
Verified bool
}
// unsafe.Sizeof → 80 bytes

That’s 24 bytes per session struct, dropped by pure reordering. No logic changed. No types changed. If you’re holding 100,000 active sessions in memory, the original layout costs roughly 2.4MB more than the packed version — and more importantly, fewer sessions fit in a cache line during iteration, which is where Ropert’s argument lands.

Run it yourself. The numbers are deterministic on any 64-bit Go runtime.

The review step that LLM output currently skips: after generating a struct, check its size with `unsafe.Sizeof`, check field alignment with `unsafe.Offsetof`, and ask whether the field ordering was chosen or defaulted. The model doesn’t reason about this. You need to.

This is exactly the kind of thing Ropert is pointing at when he talks about abstractions hiding hardware realities. The Go struct syntax abstracts away the memory layout. The LLM produces valid Go. Neither tells you about the 24 bytes of padding sitting silently between `Active` and `UserID`.

When Mechanical Sympathy Doesn’t Matter (And When It Does)

## When Mechanical Sympathy Doesn’t Matter (And When It Does)

This is the section most “think about your cache” articles skip entirely, which is why they end up reading like performance anxiety dressed up as engineering advice.

The honest answer to “is performance-first design always worth it” is: no, and anyone who tells you otherwise is optimizing for sounding rigorous rather than shipping working software.

—

### Where layout optimization is not worth the cognitive overhead

Some code genuinely does not need this treatment. Running through the list:

**CLI init and config parsing.** This runs once per process invocation. You could store your config as a linked list of YAML blobs and it wouldn’t matter. The user already waited 40ms for the shell to spawn.

**Internal tooling scripts.** If there’s no latency SLA and the audience is three engineers on your team, optimize for readability. You’ll thank yourself when you revisit it in four months.

**Services handling low request volumes with no growth trajectory.** If your service is comfortably handling traffic and you have no evidence that’s changing, you’re spending real engineering time on a hypothetical problem. The ceiling matters here — a service with a hard upper bound on users is a different beast than one in front of an unpredictable public endpoint.

**Batch jobs where I/O wait dominates.** This one trips people up because it *feels* like a CPU problem. It usually isn’t. Before you start rearranging structs or rethinking your data layout, run `perf stat` on the actual job. If your `cache-misses` rate looks fine but `block:block_rq_issue` is lighting up, you’re staring at the wrong bottleneck. Reorganizing memory layout in an I/O-bound job is theater — it makes the code harder to read and moves nothing on the wall clock.

—

### Where it is worth building in as a design constraint from day one

Some contexts make layout and access pattern decisions load-bearing. Retrofitting them later is genuinely painful, not just tedious.

**Hot path request handlers.** The code that runs on every single request deserves different treatment than the code that runs once on startup. If your request handler is allocating a new map and scanning it linearly on every call, that’s a structural problem, not a micro-optimization target. Fixing it later means changing the interface, not just the internals.

**Data pipelines processing millions of rows.** At this scale, the difference between column-oriented and row-oriented access isn’t academic. You’re either making the prefetcher’s job easy or you’re fighting it on every iteration. The pipeline structure needs to encode this from the beginning — bolting it on after the fact typically means a rewrite.

**Real-time loops.** Audio processing, game engine tick loops, robotics control loops — anything with a hard deadline measured in microseconds can’t absorb a cache miss burst in the middle of a frame. The budget is fixed. Memory layout decisions become scheduling decisions.

**Serialization and deserialization on every request.** If your ser/de code runs on every inbound and outbound message, it’s on the hot path by definition. Struct layout, field ordering, and allocation patterns here have compounding effects. A poorly laid-out struct that gets serialized a million times per second is not a micro-optimization problem — it’s a design problem.

—

### What Ropert actually says about this

Attributing this correctly matters because the nuance is easy to lose in paraphrase.

In his CppCon 2016 talk *”A System of Game Engine Components”* and more directly in his NDC Oslo 2016 talk *”Better Code: Data Structures”*, Ropert is explicit that he is not arguing for premature optimization. His framing — stated roughly in the first quarter of the NDC talk — is that the goal is **not** to optimize code you’ve already written, but to avoid writing code whose structure makes optimization impossible later. He draws a distinction between micro-optimization (fiddling with existing hot code) and structural decisions that close off optimization paths before the problem even surfaces.

His point, as stated in those talks, is that a naive object graph with ownership scattered across the heap doesn’t become cache-friendly through profiling. You profile it, discover it’s slow, and then discover the fix requires throwing away the design. The argument is for structure-aware design at the *architecture* stage, not for spending time on layout details in code that runs once.

That’s a meaningfully narrower claim than “always think about your cache.” It’s closer to: *understand what kind of code you’re writing before you write it, so you don’t design yourself into a corner on the paths that actually matter.*

The list above is just an operationalization of that distinction.

Performance Budget Template: A Copyable Artifact for Your Next Design Review

## Performance Budget Template: A Copyable Artifact for Your Next Design Review

Cut this table, paste it into your design doc, and fill in the numbers before anyone writes a line of optimized code. That ordering matters. A budget set after the optimization work is just documentation of what you already did — useful for nothing.

—

| Component / Endpoint | p50 Target (ms) | p99 Target (ms) | Max Heap Allocs / Request | Max DB Calls | Measured Baseline | Status |
|—|—|—|—|—|—|—|
| `POST /api/infer` | 120 | 400 | 50 | 2 | — | 🟡 |
| `GET /api/search` | 20 | 80 | 10 | 1 | — | 🟢 |
| Token embedding lookup | 5 | 15 | 0 | 0 | — | 🔴 |
| KV cache eviction path | 2 | 8 | 0 | 0 | — | 🟡 |
| Batch prefill pipeline | 60 | 200 | 20 | 0 | — | 🟢 |
| Auth middleware | 1 | 5 | 3 | 1 | — | 🟢 |
| DB connection checkout | 3 | 12 | 0 | 0 | — | 🔴 |

Copy the raw Markdown. Rename the rows to match your actual system. The rest of this section explains how to fill in the one column that actually requires work: **Measured Baseline**.

—

### Filling In the Measured Baseline Column

The goal is a single number per cell, measured under realistic load, not synthetic microbenchmarks. Here’s which tool reaches for which target:

**C++ paths** — Run `perf stat -e cache-misses,cache-references,instructions,cycles ./your_binary` to get CPU counter baselines for hot paths. For heap allocation counts and peak memory, `valgrind –tool=massif` gives you a full allocation graph, though it runs at roughly 20x slowdown so keep your test workload small. `heaptrack` is faster in practice and produces output you can diff between builds — attach it to a running process with `heaptrack -p ` or wrap your binary directly. Either way, export the peak allocation number and put it in the cell.

**Go paths** — `go tool pprof` against a live heap profile is the standard path. Hit your `/debug/pprof/heap` endpoint under load, then `go tool pprof -alloc_objects http://localhost:6060/debug/pprof/heap` to get alloc counts rather than retained bytes. The alloc count is what belongs in the “Max Heap Allocs / Request” column, not the byte total — the column is about GC pressure, not memory footprint. For latency numbers, `go test -bench=. -benchmem` gives you ns/op and allocs/op for isolated functions.

**Python paths** — Wrap the request handler with `tracemalloc`:

python
import tracemalloc

tracemalloc.start()
# … your request handler …
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
print(f”Peak: {peak / 1024:.1f} KiB”)

Run that under your actual request payload, not an empty fixture. Python’s allocation overhead is high by default, so the number you get will look alarming compared to Go or C++. That’s not a bug in the measurement — put the real number in the table. The budget forces an honest conversation about what “acceptable” means for your stack.

**Rust paths** — `cargo flamegraph` gets you CPU time attribution quickly. For allocation counts, pair it with `criterion` for the statistical latency baseline. A `criterion` benchmark gives you a stable p50/p99 estimate across runs that you can paste directly into the target columns. For allocation tracking, the `dhat` Valgrind tool works well for Rust: build with `–profile release` and `RUSTFLAGS=”-g”`, then run under `valgrind –tool=dhat`. The output includes total heap allocations for the measured block.

—

### The Template Is Intentionally Language-Agnostic

Every column lives at the product or system level, not inside a specific runtime. The p99 target for `POST /api/infer` doesn’t care whether the handler is Python calling a Rust extension, a pure Go service, or a C++ inference server. The contract is defined at the boundary — the network, the database connection, the user experience. That’s what makes this artifact useful across team boundaries: a frontend engineer, a backend engineer, and an ML infrastructure engineer can all read the same table and argue about the same numbers without first agreeing on what a GIL contention event costs in microseconds.

This is the cross-language argument from the rest of this piece made concrete. Ropert’s cache-miss intuitions apply to Go’s GC write barriers, Python’s reference count increments, and Rust’s allocator calls through the same underlying mechanism: the CPU stalls waiting for memory. The performance budget doesn’t need to know which mechanism fired — it just records whether the measured number landed inside the agreed boundary.

One failure mode to watch: teams fill in the target columns from intuition and leave the baseline column empty indefinitely. The status column then defaults to 🟢 everywhere because there’s nothing to compare against. Make filling in the baseline column a gate on the design review, not a follow-up action item. If you can’t measure it before shipping, you can’t know whether you broke it after.

FAQ

## FAQ

**Q: Do I need to know C++ to apply Ropert’s ideas?**

No. The hardware arguments are language-agnostic. C++ is the language Ropert uses to demonstrate the concepts — the memory hierarchy, cache line behavior, and struct layout rules are the actual subject. Go, Python, and Rust equivalents appear throughout the article precisely because the underlying machine doesn’t care what language compiled to it.

—

**Q: What’s the fastest way to find out if I have a cache problem right now?**

On Linux, `perf stat` with cache-specific counters:

bash
perf stat -e cache-misses,cache-references,L1-dcache-load-misses,L1-dcache-loads ./your_binary

The number you want is:

cache-miss rate = cache-misses / cache-references * 100

Above roughly 10% on a hot path is worth investigating. That’s not a hard threshold — a tight numerical kernel should be well under that; a pointer-chasing graph traversal hitting 30% might just be the nature of the problem. Context matters.

On macOS with Apple Silicon, `perf` isn’t available. Use Instruments via `xctrace` instead:

bash
xctrace record –template ‘CPU Counters’ –launch — ./your_binary

The “Cache Misses” instrument in the CPU Counters template gives you equivalent data. The calculation is the same.

—

**Q: Is `fieldalignment` still worth running manually in Go 1.22+?**

The workflow has changed more than the advice has. `fieldalignment` is now absorbed into `staticcheck` and most teams run it as part of CI rather than as a manual one-off check. If you’re using `golangci-lint`, it’s available as the `govet` shadow check plus the `fieldalignment` analyzer — make sure it’s enabled in your linter config rather than treating it as something you remember to run before a release.

The underlying struct reordering advice is still completely valid. Padding waste doesn’t become less real in newer Go versions. The change is just operational: you probably want this catching problems automatically at PR time rather than discovering them after the fact in a profiler.

—

**Q: Will LLM-generated code have these layout problems?**

Frequently, yes. The reasons are covered in Section 5, but the short version: LLMs are trained on code that solves correctness problems, not code that solves layout problems. The average Stack Overflow answer or tutorial repo doesn’t include struct field ordering rationale, and that’s the training distribution. So you get code that works, generates the right output, and leaks performance through padding gaps and unnecessary pointer indirection in hot paths.

The fix is a review step, not switching to a different model. Any model you pick will have this tendency because the training data does. What the article is trying to build is the habit of looking for specific patterns — pointer indirection in structs on hot paths, slice-of-interface where slice-of-concrete would do, dict/map access inside tight loops — so you can catch them in review the same way you’d catch a missing error check.

Written by Eric Woo

Lead AI Engineer & SaaS Strategist

Eric is a seasoned software architect specializing in LLM orchestration and autonomous agent systems. With over 15 years in Silicon Valley, he now focuses on scaling AI-first applications.

Cache Misses, Struct Padding, and LLM Slop: What Mathieu Ropert’s C++ Talks Actually Teach Python, Go, and Rust Devs