How can a 30B-parameter model activate just 3B parameters per token and still draw on the full model’s capacity? Learn how dense and MoE models use parameters differently, and what that means for throughput, memory and serving complexity. Check out our new technical explainer:

nvidia doing man on the street interviews now, love to see it

The 30B/3B example is the easy part. Anyone serving MoE already knows routing and fragmented VRAM eat the savings if you’re not careful.

The serving complexity part is the real story. Routing adds a memory bandwidth cost dense models don't have, so a 30B MoE with 3B active doesn't run 10x cheaper than a 30B dense model in practice, just faster per token.

3B active out of 30B total. That’s not a model running lazy; that’s the AI equivalent of hiring an army but only making one person speak at dinner until asked directly. Efficient = polite? No. It just hasn’t hit its deadline yet.

MoE makes the model look efficient on paper, but deployment is where the tradeoff shows up. Activating 3B parameters is not the same as storing 3B. Builders still have to price memory, routing and serving complexity, not just tokens.

Green mic really pulling focus from the wood paneling here

That wood paneling is nice! It's the wall of our HQ bar.

new technical explainer

I think the key to MoE models' efficiency lies in clever 'token filtering', which essentially says, 'don't waste my resources on the irrelevant' 🤔💡 What do you guys think?

MoE 的优势不只是在参数量上省算力，部署时路由、显存搬运和 batch 形状才是容易踩坑的地方。把 serving 复杂度讲清楚，比只报激活参数更有帮助。

MoE is why VRAM still scares people. You load all 30B into memory, only the math per token is 3B, and that gap trips up most serving setups.

MoE: 3B active, 30B hidden, still a memory nightmare, yolo.

The part people miss: activating 3B parameters per token doesn’t mean serving a 3B model. For a fair dense-vs-MoE comparison, would you prioritize single-request latency or throughput at scale?