Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
TopoEP: Topology-Aware Load Balancing for Expert-Parallel MoE Training
Jiacheng Zhu, Xie Zhao, Gongming Zhao +3
Dynamic routing creates severe load imbalance in large-scale expert-parallel Mixture-of-Experts (MoE) training, turning GPUs that host hot experts into stragglers. As each MoE laye…
cs.DC2026
VarioPath: Workload-Aware All-to-All Communication for PCIe GPU Clusters
Yao Fei, Jin Fang, Size Zheng +4
AlltoAllv communication is a critical primitive in distributed large-model inference, particularly for mixture-of-experts (MoE) models. The growing adoption of PCIe GPU systems for…