2 papers
cs.DC2025
DeFT: Mitigating Data Dependencies for Flexible Communication Scheduling in Distributed Training
Lin Meng, Yuzhong Sun
Communication scheduling aims to reduce communication bottlenecks in data parallel training (DP) by maximizing the overlap between computation and communication. However, existing…
cs.DC2023
Near-Linear Scaling Data Parallel Training with Overlapping-Aware Gradient Compression
Lin Meng, Yuzhong Sun, Weimin Li
Existing Data Parallel (DP) trainings for deep neural networks (DNNs) often experience limited scalability in speedup due to substantial communication overheads. While Overlapping…