Stochastic Sign Descent Methods: New Algorithms and Better Theory

Mher Safaryan, Peter Richtárik

Various gradient compression schemes have been proposed to mitigate the communication cost in distributed training of large scale machine learning models. Sign-based methods, such…