3 papers
cs.DC2026
TopoEP: Topology-Aware Load Balancing for Expert-Parallel MoE Training
Jiacheng Zhu, Xie Zhao, Gongming Zhao +3
Dynamic routing creates severe load imbalance in large-scale expert-parallel Mixture-of-Experts (MoE) training, turning GPUs that host hot experts into stragglers. As each MoE laye…
cs.DC2026
VarioPath: Workload-Aware All-to-All Communication for PCIe GPU Clusters
Yao Fei, Jin Fang, Size Zheng +4
AlltoAllv communication is a critical primitive in distributed large-model inference, particularly for mixture-of-experts (MoE) models. The growing adoption of PCIe GPU systems for…
cs.NI2026
HOCCL: Offloading Collective Communication from GPU Cores to Accelerate Distributed Training
Yao Fei, Gongming Zhao, Hongli Xu +5
Large language model training involves massive computation on GPU streaming multiprocessors (SMs), the primary compute units of GPUs. Since SMs host specialized accelerators such a…