ByteComp: Revisiting Gradient Compression in Distributed Training

Zhuang Wang, Haibin Lin, Yibo Zhu +1

Gradient compression (GC) is a promising approach to addressing the communication bottleneck in distributed deep learning (DDL). However, it is challenging to find the optimal comp…