Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression

Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +2

Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such bo…