3 papers
cs.DC2026
AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
Ling Chen, Houming Wu, Wenjie Yu
Pipeline parallelism is essential for large-scale model training, but existing asynchronous approaches often degrade convergence due to parameter mismatch between forward and backw…
cs.LG2025
TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training
Houming Wu, Ling Chen
Training large language models (LLMs) is fundamentally constrained by limited device memory and costly inter-device communication. Although pipeline parallelism alleviates memory p…
cs.LG2024
BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
Houming Wu, Ling Chen, Wenjie Yu
With the increasing scale of models, the need for efficient distributed training has become increasingly urgent. Recently, many synchronous pipeline parallelism approaches have bee…