TAOT: Topology-Aware Optimal Transport for Dynamic Expert Replica Placement in MoE Training

Lingyun Zhang, Henghua Zhang, Shilei Gu +5

Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs), yet its dynamic routing causes severe load imbalance in expert-parallel training. E…