Guides · Basics · 6 min read
What is a mixture-of-experts (MoE) model, and why does it run so fast?
Some of the most popular local models in 2026 have names like Qwen3 30B-A3B, Gemma 4 26B-A4B and gpt-oss 20B. They are mixture-of-experts models, usually shortened to MoE, and they behave very differently from traditional "dense" models on your hardware. This guide explains what MoE means in plain language and how to use it to get more out of your GPU, Mac or CPU.
Dense vs mixture of experts, in one paragraph
A dense model uses every one of its parameters to produce each word. A 32B dense model does 32 billion parameters' worth of work per word. A mixture-of-experts model is split into many smaller "experts", and for each word a router picks only a few of them. Qwen3 30B-A3B holds 30.5 billion parameters in total, but only about 3.3 billion are active for any given word. That is what the "A3B" in the name means: about 3 billion active.
Why MoE models are fast
When you run a model locally, speed is limited by how fast your hardware can read the model from memory for each new word. A dense model has to read all of its weights every time. An MoE model only reads the active experts. Fewer bytes per word means more words per second.
Here is what that looks like on real hardware, from our checker at 8K context:
| Model | Type | Total / active | RTX 3090 | Mac M4 Pro 48 GB |
|---|---|---|---|---|
| Qwen3.6 27B | Dense | 27.5B / 27.5B | ~37 tok/s | ~6 tok/s |
| Qwen3 30B-A3B | MoE | 30.5B / 3.3B | ~123 tok/s | ~26 tok/s |
| Gemma 4 31B | Dense | 31B / 31B | Offload, ~5 tok/s | ~7 tok/s |
| Gemma 4 26B-A4B | MoE | 25.8B / 4B | ~110 tok/s | ~18 tok/s |
At roughly the same size, the MoE models are three to four times faster.
The catch: MoE models still need the memory
An MoE model is fast, but it is not small. All the experts must be loaded, because the router can pick any of them for the next word. Qwen3 30B-A3B needs about 20.6 GB at Q4 with 8K context, about the same as a dense 30B model. So MoE does not let a 30B model fit on a 12 GB card.
What it does change is what happens when the model does not fit. Because only a small part is used per word, an MoE model with half its weights in system RAM still runs at a usable speed:
- On an 8 GB card with 32 GB of RAM, Qwen3 30B-A3B runs at roughly 10 tokens per second with offload. A dense model of the same size drops to about 2.
- On a CPU with no GPU at all, gpt-oss 20B and Qwen3 30B-A3B run at about 10 tokens per second, more than twice as fast as a dense 14B model. See the CPU-only guide.
Is an MoE model as smart as a dense one?
Roughly speaking, an MoE model's knowledge scales with its total size, while its speed scales with its active size. In practice, Qwen3 30B-A3B answers about as well as a good dense 14B to 32B model, depending on the task, while running like a 3B model. Dense models still tend to be a little more consistent on long, careful tasks at the same total size. For most people running models at home, the speed advantage wins.
Popular MoE models for local use
| Model | Total / active | Memory at Q4, 8K | Good on |
|---|---|---|---|
| gpt-oss 20B | 21B / 3.6B | 14 GB | 16 GB cards, CPUs with 32 GB RAM |
| Gemma 4 26B-A4B | 25.8B / 4B | 18.9 GB | 24 GB cards, Macs with 24 GB+ |
| Qwen3 30B-A3B | 30.5B / 3.3B | 20.6 GB | 24 GB cards, Macs with 32 GB+ |
| Qwen3-Coder 30B-A3B | 30.5B / 3.3B | 20.6 GB | Coding on 24 GB |
| Qwen3.6 35B-A3B | 35B / 3B | 23.5 GB | 32 GB cards, Macs with 36 GB+ |
| gpt-oss 120B | 117B / 5.1B | about 77 GB | Macs with 96 to 128 GB |
When to pick MoE
- Your model barely fits, or does not fit: pick MoE. It degrades gracefully with offload.
- You are on a Mac or CPU: pick MoE. Lower bandwidth hurts dense models most.
- You want speed for coding agents or long chats: pick MoE.
- You have plenty of VRAM and want maximum consistency: a dense model of the same size is a fine choice.
Filter by your hardware in the checker: MoE models are marked with an "MoE" tag in the results.