BLUF On my M2 Pro Mac mini, the MLX backend consistently beats llama.cpp on both load time and generation speed — but the size of the gap depends heavily on the model. For Qwen3.6 27B (dense), MLX lands at 9.3 tok/s generation vs 7.7 tok/s for llama.cpp. For Gemma4 26B (MoE), the gap widens to … Read more
BLUF On my 32 GB Mac mini (Apple M2 Pro), qwen3-coder:30b — a 30B Mixture-of-Experts model with roughly 3B active parameters — hit 43.3 tok/s generation speed, well ahead of the dense qwen2.5-coder:32b at 6.6 tok/s and qwen2.5-coder:14b at 15.1 tok/s. If raw generation throughput is your priority on constrained hardware, the MoE architecture makes … Read more
The short version: On a Mac mini (Apple M2 Pro, 32 GB), the fastest model I tested was also the biggest — Qwen3-Coder 30B, a Mixture-of-Experts model with only ~3B active parameters, ran at 42.8 tokens/sec, beating a plain 8B model. Total parameter count barely predicts speed on this hardware; active parameters do. And the much-hyped … Read more