MoE vs dense: why a 30B-A3B runs near 8B speed
Mixture-of-experts models activate only a slice of their weights per token, so they decode about as fast as a small model — but you still need memory for the whole thing.
4 min read Reviewed July 2026
TL;DR
Model names like 30B-A3B describe a mixture-of-experts (MoE) model: roughly 30B parameters in total, with only about 3B active for any one token. That split is why an MoE decodes at close to small-model speed — but every expert must stay resident, so you still need memory for the whole 30B. Budget memory against the total count and speed expectations against the active count.
Model names like 30B-A3B or 235B-A22B carry two numbers, and they matter for very different reasons: one sets the memory you need, the other sets the speed you get.
What's the difference between dense and MoE?
A dense model uses every weight to produce every token. A 13B dense model reads all 13B parameters per token, every single token.
An MoE model splits much of the network into many "experts" and adds a small router that, for each token, picks only a few experts to run. The rest sit idle for that token — so only a fraction of the weights are active at any step.
What does the A-notation mean?
The A number is the count of active parameters per token, in billions:
- 30B-A3B — about 30B parameters total, but only ~3B active per token.
- 235B-A22B — about 235B total, ~22B active per token.
The total tells you what has to fit in memory; the active count tells you how much work each generated token costs.
Fast to decode, heavy to hold
Token generation is bound by how many bytes you read per token (Bandwidth vs compute). Because an MoE only reads its active weights each token, a 30B-A3B streams roughly the bytes of a ~3B model — so it decodes at close to small-model speed, well above what a 30B dense model would manage.
Here is the catch: any token can route to any expert, so all the experts have to be resident in memory. You must fit the whole ~30B (at your quant) in VRAM or unified memory, even though only ~3B run per token.
MoE: memory of the big number, decode speed near the small number.
Plan your memory budget against the total size (see the VRAM guide), and your speed expectations against the active size.
MoE (30B-A3B)
- Memory needed: the full ~30B must be resident, at your quant
- Bytes read per token: only the ~3B active weights
- Decode speed: close to what a ~3B-class model manages
- Prefill: touches more of the network than a single decode step — costlier than the active count alone suggests
Dense (32B)
- Memory needed: all 32B, at your quant
- Bytes read per token: all 32B weights, every token
- Decode speed: big-model speed — far slower per token
- Prefill: compute-bound, in line with the model's full size
Trade-offs to keep honest
MoE is not a free lunch, and two caveats stop the headline from over-promising:
- Prompt processing touches more of the network than a single decode step, so the prefill cost is not as small as the active-parameter count alone suggests (pp vs tg).
- MoE buys more stored knowledge per unit of compute, but a 3B-active model is not automatically as capable as a 30B-dense model on every task — treat active and total counts as two different things, not one.
Is a 30B-A3B as smart as a 30B dense model?
Not automatically. The MoE stores more knowledge per unit of compute, but only ~3B parameters do the work on any one token — on some tasks it behaves more like a very well-read small model. Treat total and active counts as two different measures, and test on your own tasks.
Can I run a 30B-A3B on hardware that only fits a 3B model?
No. The memory question is set by the total: all ~30B parameters must be resident, because any token can route to any expert. The active count buys you speed, not a smaller footprint.