BLUF
On my 32 GB M2 Pro Mac mini, Apple MLX models loaded noticeably faster and generated tokens faster than their llama.cpp equivalents in every test — for example, gemma4:26b hit 38.7 tok/s on MLX vs 30.1 tok/s on llama.cpp, and qwen3:27b hit 9.3 tok/s vs 7.7 tok/s. That said, the generation gains are real but modest; the bigger story is load time and prompt throughput, where the gap is more obvious.
Why I Ran This
The “MLX is way faster” claim floats around homelab and self-hosted AI communities constantly, but I rarely see it pinned to specific numbers on specific hardware. Ollama now ships MLX-backed model variants alongside the default llama.cpp ones, so I finally had an easy apples-to-apples setup on a single machine. Today I ran both backends against the same prompt and recorded what actually came out.
Test Setup
- Hardware: Mac mini, Apple M2 Pro, 32 GB unified memory, macOS
- Ollama version: 0.32.1
- Models: qwen3.6:27b (dense) and gemma4:26b (MoE), each in default llama.cpp and MLX variants
- Method: Single fixed prompt,
num_predict=256, temperature 0.3, thinking off, warm average of 2 runs per model - Quantizations: Default for each variant — I did not pin matching quant levels across backends
Honest Limitations
This is throughput-only testing. I am measuring tokens per second and load time, not output quality, coherence, or accuracy — different quantizations between the llama.cpp and MLX builds mean the two outputs are not necessarily identical. One machine, one prompt, two runs each. Treat these as directional data points, not a definitive study. Also: the gemma4:26b MLX prompt throughput figure (2,507,500.1 tok/s) looks like a measurement artifact or Ollama reporting quirk — I am including it as recorded but would not read into that specific number.
Results
| Model | Backend | Load Time (s) | Generation (tok/s) | Prompt Throughput (tok/s) |
|---|---|---|---|---|
| qwen3.6:27b (dense) | llama.cpp | 20.4 | 7.7 | 167.4 |
| qwen3.6:27b (dense) | MLX | 9.0 | 9.3 | 308.6 |
| gemma4:26b (MoE) | llama.cpp | 16.8 | 30.1 | 513.6 |
| gemma4:26b (MoE) | MLX | 7.9 | 38.7 | 2,507,500.1 (likely artifact — see note) |
What I Actually Take Away From This
Load time: MLX wins clearly on both models. qwen3.6:27b loaded in 9.0 s on MLX vs 20.4 s on llama.cpp. gemma4:26b loaded in 7.9 s on MLX vs 16.8 s on llama.cpp. If you are hot-swapping models frequently, this compounds fast.
Generation speed: MLX is faster, but the margin is real-but-not-dramatic. qwen3.6:27b goes from 7.7 tok/s to 9.3 tok/s. gemma4:26b goes from 30.1 tok/s to 38.7 tok/s. You will feel the difference more on the MoE model at interactive speeds. On the dense 27B at under 10 tok/s either way, the experience is similarly slow.
Prompt throughput: For qwen3.6:27b, MLX posts 308.6 tok/s vs 167.4 tok/s on llama.cpp — meaningful if you are doing long-context or RAG work. The gemma4:26b MLX figure I recorded is implausibly high and I suspect it is an Ollama 0.32.1 reporting bug for that model/backend combination. I am leaving it in the table as-is so you can see exactly what I captured.
Is MLX worth the hype? Partially. Load times and prompt throughput show genuine gains. Generation throughput improvements are real but modest — nobody is going to feel like they upgraded their hardware. The hype tends to lead with the best-case numbers; my data says “yes, faster, but keep expectations grounded.”
My Current Take
On a 32 GB M2 Pro, I will default to MLX variants when they are available, mainly for the load time win and the prompt throughput boost. For pure generation speed on a dense 27B, neither backend is what I would call fast — 7.7 vs 9.3 tok/s is a real difference but both feel slow for interactive use. Gemma4:26b at 30–38 tok/s is where MLX starts feeling meaningfully better in practice.
If you have run similar comparisons on a different Apple Silicon chip or with different Ollama versions, drop your numbers in the comments — I am especially curious whether M3 Ultra or M2 Max changes the picture.
