Enhancing SignSGD: Small-Batch Convergence Analysis and a Hybrid Switching Strategy

Haoran Chen, Wentao Wang

SignSGD compresses each stochastic gradient coordinate to a single bit, offering substantial memory and communication savings, but its 1-bit quantization removes magnitude informat…