Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression

Boao Kong, Xu Huang, Yuqi Xu +3

Pipeline-parallel distributed optimization is essential for large-scale machine learning but is challenged by significant communication overhead from transmitting high-dimensional…