Asynchronous decentralized accelerated stochastic gradient descent

Abstract

In this work, we introduce an asynchronous decentralized accelerated stochastic gradient descent type of method for decentralized stochastic optimization, considering communication and synchronization are the major bottlenecks. We establish $\mathcal{O}(1/\epsilon )$ (resp., $\mathcal{O}(1/\sqrt{\epsilon })$) communication complexity and $\mathcal{O}(1/{\epsilon }^2)$ (resp., $\mathcal{O}(1/\epsilon )$) sampling complexity for solving general convex (resp., strongly convex) problems.

References in corpus (2)

• Asynchronous Distributed Optimization using a Randomized Alternating Direction Method of Multipliers
• Decentralized Consensus Optimization with Asynchrony and Delays

Cited by in corpus (2)

• Robust and Communication-Efficient Collaborative Learning
• Accelerated Sparsified SGD with Error Feedback