IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method
arXiv:2006.06733
Abstract
We introduce a framework for designing primal methods under the decentralized optimization setting where local functions are smooth and strongly convex. Our approach consists of approximately solving a sequence of sub-problems induced by the accelerated augmented Lagrangian method, thereby providing a systematic way for deriving several well-known decentralized algorithms including EXTRA arXiv:1404.6264 and SSDA arXiv:1702.08704. When coupled with accelerated gradient descent, our framework yields a novel primal algorithm whose convergence rate is optimal and matched by recently derived lower bounds. We provide experimental results that demonstrate the effectiveness of the proposed algorithm on highly ill-conditioned problems.
References in corpus (7)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
- Optimal algorithms for smooth and strongly convex distributed optimization in networks
- Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
- A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates
- Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
- Bregman Augmented Lagrangian and Its Acceleration
Cited by in corpus (6)
- A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning
- Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
- Accelerated Gradient Tracking over Time-varying Graphs for Decentralized Optimization
- Cross-Gradient Aggregation for Decentralized Learning from Non-IID data
- An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints
- ADMM-based Distributed State Estimation for Power Systems: Evaluation of Performance