On the Limits of Momentum in Decentralized and Federated Optimization

Riccardo Zaccone, Sai Praneeth Karimireddy, Carlo Masone

Recent works have explored the use of momentum in local methods to enhance distributed SGD. This is particularly appealing in Federated Learning (FL), where momentum intuitively ap…