Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Models

Heng Zhu, Harsh Vardhan, Arya Mazumdar

In distributed training of machine learning models, gradient descent with local iterative steps, commonly known as Local (Stochastic) Gradient Descent (Local-(S)GD) or Federated av…