What Actually Predicts Local LLM Speed on a 32 GB Mac (2026): I Benchmarked 7 Models

by

The short version: On a Mac mini (Apple M2 Pro, 32 GB), the fastest model I tested was also the biggest — Qwen3-Coder 30B, a Mixture-of-Experts model with only ~3B active parameters, ran at 42.8 tokens/sec, beating a plain 8B model. Total parameter count barely predicts speed on this hardware; active parameters do. And the much-hyped Apple MLX backend? Real, but modest here: about 14–19% faster generation and ~2× faster model loading — not the “93% faster” you’ll read elsewhere.

The setup

This is the single machine I run my whole self-hosted stack on — not a datacenter GPU:

  • Hardware: Mac mini, Apple M2 Pro (10-core CPU: 6 performance + 4 efficiency), 32 GB unified memory
  • OS / runtime: macOS 26.4.1, Ollama 0.32.1, default (4-bit) tags unless noted
  • Why unified memory matters: on Apple Silicon there’s no separate VRAM — the model shares the same 32 GB as everything else, so generation is largely memory-bandwidth bound. This box also runs n8n in Docker and an automation engine at the same time.

Method (reproducible)

  • One fixed ~200-word prompt, num_predict = 256, temperature = 0.3, thinking disabled.
  • Per model: one cold run (includes load) + two warm runs; the generation figure is the average of the two warm runs.
  • Numbers come from Ollama’s own /api/generate timing fields, not a stopwatch.
ollama run qwen3-coder:30b --verbose "Explain what a reverse proxy is in ~200 words."
# read the eval rate (tokens/s) it prints

Limitations, stated plainly: single prompt, one machine, default quantizations, short 256-token output. This measures short-generation throughput on this Mac — not quality, not long context, not your hardware.

The results

Model Architecture Backend Load (s) Generation (tok/s) Prompt (tok/s)
llama3.1:8b dense 8B llama.cpp 5.1 28.1 1425
phi4:14b dense 14B llama.cpp 3.7 15.7 804
qwen3.6:27b dense 27B llama.cpp 15.1 8.3 188
qwen3.6:27b dense 27B MLX 8.1 9.9 271
gemma4:26b MoE (≈4B active) llama.cpp 15.4 32.1 521
gemma4:26b MoE (≈4B active) MLX 7.5 36.6 — *
qwen3-coder:30b MoE (≈3B active) llama.cpp 19.4 42.8 1423

* one MLX prompt-processing reading was an obvious measurement artifact, so I’ve left it out rather than print a bogus number.

Finding 1: active parameters, not total size, decide speed

Look at the dense models alone and the story is predictable — bigger is slower, cleanly: 8B → 28 t/s, 14B → 16 t/s, 27B → 8 t/s.

Now add the Mixture-of-Experts models. A MoE has a large total parameter count but only activates a small slice per token:

  • gemma4:26b (≈4B active): 32 t/s — a 26B model faster than the 8B.
  • qwen3-coder:30b (≈3B active): 43 t/s — the largest model in the test, and the fastest generator, period.

That’s the whole 2026 efficiency story in two rows. If you’ve been choosing local models by their headline size, you’ve been using the wrong number. Look at active parameters.

Finding 2: MLX helps — but temper the hype

You’ll see the claim that Ollama’s Apple MLX backend is “~93% faster” on Apple Silicon. On this machine, at these quantizations, it wasn’t:

  • qwen3.6:27b (dense): 8.3 → 9.9 t/s generation (+19%), and it loaded in 8.1 s vs 15.1 s (~2× faster).
  • gemma4:26b (MoE): 32.1 → 36.6 t/s generation (+14%), loading in 7.5 s vs 15.4 s (~2× faster).

So MLX is worth using — it’s a free speedup and it cuts load time roughly in half — but on my setup the generation gain was 14–19%, not 93%. Headline benchmark numbers are often measured on a specific model, quant, and context; yours will differ. Measure it before you believe it.

What I actually run

  • Daily driver, and the surprise winner: qwen3-coder:30b. Fastest generation in the test and a capable 256K-context coder. On a 32 GB Mac, a well-built MoE is hard to beat.
  • Great general-purpose pick: gemma4:26b (grab the -mlx tag for the faster loads).
  • Avoid for interactive use: dense 27B (qwen3.6:27b at 8 t/s) — fine for a background job, painful to watch.
  • Keep models warm with Ollama’s keep_alive; load time only bites on the first request after an idle gap (and MLX halves even that).

Bottom line

On a 32 GB Apple Silicon machine in 2026, two things predict local-LLM speed far better than model size: how many parameters are actually active (favor MoE), and whether you’re on the MLX backend. The biggest model I tested was the fastest — because it only wakes up 3B parameters at a time. Pull two tags, run the one-liner above on your own box, and see for yourself.

Measured 2026-08-12 on a Mac mini (Apple M2 Pro, 32 GB), macOS 26.4.1, Ollama 0.32.1. If you run the same test, I’d genuinely like to see your numbers.

Leave a Comment