Inference 0/0

Inference

Mixture of experts

Layers in which a router sends each token to a few of many expert networks, with capacity limits, load-balancing losses and all-to-all communication between devices. Large open models such as DeepSeek-V3 and Mixtral are built this way.

0 / 8 solved

Questions

0/8 solved
Question Tags Difficulty Success
1. Pick Your Experts
moerouting +1
Medium —
2. Capacity Overflow
moetoken-dropping +1
Hard —
3. Traffic Jam at Expert Seven
moeexpert-parallelism +2
Hard —
4. Z-Loss Guardrail
moerouter +1
Hard —
5. The Overflow Bucket
moecapacity-factor +1
Insane 40%
6. All-to-All Wall
moeexpert-parallelism +1
Insane —
7. Bias, Not Backprop
moeload-balancing
Insane —
8. Sparse Compute, Dense Memory
moeexpert-parallelism +1
Insane —