163 citations · 496 across the 40 of their papers we have counts for
3 papers · 1 filter
Stochastic Gradient Push for Distributed Deep Learning
Mahmoud Assran, Nicolas Loizou, Nicolas Ballas +1
Distributed data-parallel algorithms aim to accelerate the training of deep neural networks by parallelizing the computation of large mini-batch gradient updates across multiple no…
Provably Accelerated Randomized Gossip Algorithms
Nicolas Loizou, Michael Rabbat, Peter Richtárik
In this work we present novel provably accelerated gossip algorithms for solving the average consensus problem. The proposed protocols are inspired from the recently developed acce…
Accelerated Gossip via Stochastic Heavy Ball Method
Nicolas Loizou, Peter Richtárik
In this paper we show how the stochastic heavy ball method (SHB) -- a popular method for solving stochastic convex and non-convex optimization problems --operates as a randomized g…