TerraceMoE: A Cost Model for Hierarchical MoE All-to-All Communication

Weicheng Xue, Bingqiang Wang, Li Yuan +2

Hierarchical two-hop dispatch can reduce slow-fabric traffic in expert-parallel Mixture-of-Experts training, but it adds a second collective and an arrival-side operator chain. We…