BLUF
On my M2 Pro Mac mini, the MLX backend consistently beats llama.cpp on both load time and generation speed — but the size of the gap depends heavily on the model. For Qwen3.6 27B (dense), MLX lands at 9.3 tok/s generation vs 7.7 tok/s for llama.cpp. For Gemma4 26B (MoE), the gap widens to 38.5 tok/s vs 30.1 tok/s. Load times drop noticeably with MLX across both models.
Why I Ran This
“MLX is faster on Apple Silicon” is a claim I see constantly in homelab and local-LLM circles. It’s plausible — MLX is Apple’s own framework, purpose-built for unified memory and the Neural Engine. But I wanted actual numbers from my own machine, not marketing copy. So I ran the same prompt on matched model pairs: one llama.cpp-backed, one MLX-backed, same quantization tier where available.
My Setup
- Hardware: Mac mini, Apple M2 Pro, 32 GB unified memory, macOS
- Inference engine: Ollama 0.32.1
- Models tested: qwen3.6:27b / qwen3.6:27b-mlx, gemma4:26b / gemma4:26b-mlx
- Prompt: Single fixed prompt, same for all runs
- Settings: num_predict=256, temperature=0.3, thinking off
- Averaging: Warm average of 2 runs per model (first cold run discarded)
Results
| Model | Backend | Load Time (s) | Generation (tok/s) | Prompt Eval (tok/s) |
|---|---|---|---|---|
| Qwen3.6 27B (dense) | llama.cpp | 15.1 | 7.7 | 166.5 |
| Qwen3.6 27B (dense) | MLX | 8.3 | 9.3 | 295.6 |
| Gemma4 26B (MoE) | llama.cpp | 16.3 | 30.1 | 509.4 |
| Gemma4 26B (MoE) | MLX | 7.6 | 38.5 | 2,934,395.3 ⚠️ |
What the Numbers Actually Show
Generation speed: MLX wins on both models. For Qwen3.6 27B, it’s 9.3 tok/s vs 7.7 tok/s — a real but modest improvement. For Gemma4 26B (MoE), it’s 38.5 tok/s vs 30.1 tok/s, which is a more meaningful day-to-day difference. Neither number is going to shock anyone expecting a dramatic transformation, but MLX does move the needle.
Load time: This is where MLX actually shines most cleanly. Qwen3.6 27B loads in 8.3 s with MLX vs 15.1 s with llama.cpp. Gemma4 26B loads in 7.6 s vs 16.3 s. If you’re frequently swapping models, that adds up fast.
Prompt eval speed — the Gemma4 MLX outlier: I have to be straight with you: the Gemma4 26B MLX prompt eval number of 2,934,395.3 tok/s is almost certainly a measurement artifact, not a real throughput figure. Ollama can report distorted prompt eval numbers when the prompt is extremely short or the tokenization path behaves unexpectedly with certain backends. I’m including it for full transparency, but do not interpret it as a real-world result. The llama.cpp figure of 509.4 tok/s for the same model is plausible; the MLX figure is not. Flag it, don’t trust it.
Honest take on the hype: MLX is genuinely faster here, but if you were expecting a dramatic leap, the generation numbers are modest on a dense model like Qwen3.6 27B. The gains are more visible on the MoE architecture (Gemma4), and load times improve across the board. Worth switching if you’re already using Ollama and MLX variants are available — but it’s not a night-and-day transformation for every model.
Limitations
- Single fixed prompt only — throughput numbers can shift with different prompt lengths and content
- num_predict=256 caps output; results may differ on longer generations
- Warm average of just 2 runs — more runs would tighten confidence
- Default quantizations used; I did not verify that both backends are using identical quant levels
- One machine only — your M1, M2 Max, M3 Ultra results will differ
- This measures throughput only, not output quality
- The Gemma4 MLX prompt eval number is almost certainly a measurement artifact and should be disregarded
If you’ve run similar MLX vs llama.cpp benchmarks on your own Apple Silicon machine — especially on M1 Pro, M2 Max, or M3-series — drop your numbers in the comments.
