Conditional Capacity and Routing in Mixture-of-Experts Particle Transformers

Kaushik Pendiyala, Haris Zia, Trevin Lee +8

Mixture-of-Experts (MoE) models can increase parameter capacity without proportionally increasing active computation, but it is unclear how this trade-off behaves in particle-physi…