Mixture-of-Experts Models in Vision: Routing, Optimization, and Generalization

Adam Rokah, Daniel Veress, Caleb Caulk +1

Mixture-of-Experts (MoE) architectures enable conditional computation by routing inputs to multiple expert subnetworks and are often motivated as a mechanism for scaling large lang…