2 papers
cs.DC2025
FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion
Zhuoran Zhu, Chunyang Zhu, Hao Lin +9
Large-scale Mixture-of-Experts (MoE) models rely on \emph{expert parallelism} for efficient training and inference, which splits experts across devices and necessitates distributed…
cs.LG2025
STAlloc: Enhancing Memory Efficiency in Large-Scale Model Training with Spatio-Temporal Planning
Zixiao Huang, Junhao Hu, Hao Lin +9
The rapid scaling of large language models (LLMs) has significantly increased GPU memory pressure, which is further aggravated by training optimization techniques such as virtual p…