Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
TAOT: Topology-Aware Optimal Transport for Dynamic Expert Replica Placement in MoE Training
Lingyun Zhang, Henghua Zhang, Shilei Gu +5
Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs), yet its dynamic routing causes severe load imbalance in expert-parallel training. E…
cs.DC2025
Astra: Efficient and Money-saving Automatic Parallel Strategies Search on Heterogeneous GPUs
Peiran Wang, Haibing Li, Fu Haohan +3
In this paper, we introduce an efficient and money-saving automatic parallel strategies search framework on heterogeneous GPUs: Astra. First, Astra searches for the efficiency-opti…
cs.DC2024
Mao: Machine learning approach for NUMA optimization in Warehouse Scale Computers
Yueji Liu, Jun Jin, Wenhui Shu +2
Non-Uniform Memory Access (NUMA) architecture imposes numerous performance challenges to today's cloud workloads. Due to the complexity and the massive scale of modern warehouse-sc…