Inference 0/0

Inference

Mixture of experts

Layers in which a router sends each token to a few of many expert networks, with capacity limits, load-balancing losses and all-to-all communication between devices. Large open models such as DeepSeek-V3 and Mixtral are built this way.

0 / 8 solved