An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models

Arjun Ashok Rao, Hoi-To Wai

This paper considers decentralized optimization with application to machine learning on graphs. The growing size of neural network (NN) models has motivated prior works on decentra…