Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
HAPMoE: Heterogeneity-Aware Automatic Parallelism Planning for Mixture-of-Experts Models Training
Mengyuan Fan, Peizhuang Cong, Zixiao Huang +7
As model sizes continue to scale, distributed training has become inevitable. Automatic parallelization techniques can derive efficient training parallelism strategies at low cost…
cs.DC2025
FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion
Zhuoran Zhu, Chunyang Zhu, Hao Lin +9
Large-scale Mixture-of-Experts (MoE) models rely on \emph{expert parallelism} for efficient training and inference, which splits experts across devices and necessitates distributed…