Inference
Mixture of experts
Layers in which a router sends each token to a few of many expert networks, with capacity limits, load-balancing losses and all-to-all communication between devices. Large open models such as DeepSeek-V3 and Mixtral are built this way.
Questions
Question Difficulty Success