MoE routing strategies shape model quality, balance and serving costs
Mixture-of-Experts models use a router for each token, selecting a small number of expert networks instead of running all of them. The choice affects model quality, workload balance and the communication costs of multi-GPU serving. Mixtral 8x7B, for example, exposes 47B parameters but activates 13B for each token.