BLUF
On my 32 GB Mac mini (M2 Pro), dense models scale predictably — going from 8B to 14B drops generation speed from 27.9 tok/s to 15.6 tok/s. But MoE architecture throws that rule out: qwen3-coder:30b hits 42.8 tok/s, faster than every other model I tested, including the 8B dense one.
The Setup
Hardware: Mac mini, Apple M2 Pro, 32 GB unified memory, macOS, Ollama 0.32.1. I ran each model against a single fixed prompt with num_predict=256, temperature 0.3, thinking off. I discarded the first run and averaged two warm runs. All models used their default Ollama quantizations. This measures throughput only — not output quality, reasoning depth, or real-world task performance.
Results
| Model | Load (s) | Generation (tok/s) | Prompt eval (tok/s) |
|---|---|---|---|
| llama3.1:8b (dense 8B) | 4.6 | 27.9 | 1380.7 |
| phi4:14b (dense 14B) | 8.8 | 15.6 | 806.9 |
| gpt-oss:20b | 14.9 | 35.0 | 1215.1 |
| gemma4:26b (MoE, ~4B active) | 13.9 | 32.2 | 536.6 |
| gemma4:31b (MoE) | 19.3 | 6.5 | 112.2 |
| qwen3-coder:30b (MoE, ~3B active) | 19.1 | 42.8 | 2243.3 |
What the Numbers Actually Show
Dense Models: Bigger Is Slower, Predictably
The dense model story is straightforward. Llama3.1:8b generates at 27.9 tok/s with a 4.6 s load time. Step up to phi4:14b and generation drops to 15.6 tok/s, load climbs to 8.8 s. More weights, more memory bandwidth consumed per token, slower output. There are no surprises here and no tricks.
gpt-oss:20b is the one dense outlier worth flagging — 35.0 tok/s generation despite being the heaviest dense model I tested. I don’t have insight into its internal architecture beyond the label, so I’ll just report the number and note it doesn’t fit the simple “bigger = slower” pattern for dense models either.
MoE: Parameter Count Is the Wrong Number to Watch
This is where it gets interesting. gemma4:26b carries a 26B parameter count but only activates roughly 4B parameters per forward pass. It generates at 32.2 tok/s — faster than phi4:14b at 15.6 tok/s, despite having a nominally larger model file on disk. The memory bandwidth the chip actually burns per token tracks active parameters, not total parameters.
qwen3-coder:30b takes this further: 30B total, ~3B active, and it produces 42.8 tok/s generation and 2243.3 tok/s prompt evaluation — the fastest numbers in this entire test. A 30B model that out-generates an 8B dense model is not an intuitive result if you’ve been reasoning about model size in total parameter terms.
gemma4:31b is the cautionary counterpoint. Also a MoE model, also carrying a large parameter count, but it generates at only 6.5 tok/s with a 112.2 tok/s prompt eval — the slowest in the set by a wide margin. MoE label alone doesn’t guarantee speed. Something about this model’s routing, quantization, or memory footprint is hitting my 32 GB ceiling hard. Load time is 19.3 s, and the generation throughput suggests it’s not fitting as cleanly into unified memory as qwen3-coder:30b does.
Honest Limitations
- Single fixed prompt. A different prompt length or content could shift prompt eval numbers substantially.
num_predict=256— short generation window. Sustained throughput over longer outputs may differ.- Two warm runs averaged. Small sample size; thermal state and memory pressure vary run to run.
- Default quantizations only. A lower quant of gemma4:31b might fit better and change its numbers entirely.
- One machine, one benchmark angle: throughput. Output quality is not measured here at all.
- “Active parameters” figures for MoE models come from model documentation, not direct measurement.
The Takeaway
On a 32 GB M2 Pro, dense model scaling behaves as expected — every billion parameters costs you throughput. But MoE models make total parameter count a misleading proxy for speed. qwen3-coder:30b at 42.8 tok/s generation is the clearest example: a 30B model you can run faster than an 8B dense model, if the architecture and quantization fit your memory well. gemma4:31b at 6.5 tok/s is the reminder that not every MoE model lands that cleanly on 32 GB.
If you’re shopping for a local model by parameter count alone, these numbers suggest you should be shopping by active parameter count and memory fit instead.
Running your own benchmarks on different hardware or with different quantizations? Drop your numbers in the comments — I’d like to see how this scales on 64 GB or on Intel UHD machines.
