Improved Convergence in Parameter-Agnostic Error Feedback through Momentum

Abdurakhmon Sadiev, Yury Demidovich, Igor Sokolov +3

Communication compression is essential for scalable distributed training of modern machine learning models, but it often degrades convergence due to the noise it introduces. Error…