Multi-Agent Distributed Optimization via Inexact Consensus ADMM
arXiv:1402.6065 · doi:10.1109/TSP.2014.2367458
Abstract
Multi-agent distributed consensus optimization problems arise in many signal processing applications. Recently, the alternating direction method of multipliers (ADMM) has been used for solving this family of problems. ADMM based distributed optimization method is shown to have faster convergence rate compared with classic methods based on consensus subgradient, but can be computationally expensive, especially for problems with complicated structures or large dimensions. In this paper, we propose low-complexity algorithms that can reduce the overall computational cost of consensus ADMM by an order of magnitude for certain large-scale problems. Central to the proposed algorithms is the use of an inexact step for each ADMM update, which enables the agents to perform cheap computation at each iteration. Our convergence analyses show that the proposed methods converge well under some convexity assumptions. Numerical results show that the proposed algorithms offer considerably lower computational complexity than the standard ADMM based distributed optimization methods.
submitted to IEEE Trans. Signal Processing; Revised April 2014 and August 2014
References in corpus (1)
Cited by in corpus (85)
- Multi-Agent Distributed Optimization via Inexact Consensus ADMM
- A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates
- Accelerated Distributed Nesterov Gradient Descent
- A Proximal Dual Consensus ADMM Method for Multi-Agent Constrained Optimization
- Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
- Distributed Nash Equilibrium Seeking under Partial-Decision Information via the Alternating Direction Method of Multipliers
- Distributed Optimization for Smart Cyber-Physical Networks
- Distributed Radio Interferometric Calibration
- Distributed Learning in the Non-Convex World: From Batch to Streaming Data, and Beyond
- Asynchronous Distributed ADMM for Large-Scale Optimization- Part II: Linear Convergence Analysis and Numerical Performance
- Distributed Non-Convex First-Order Optimization and Information Processing: Lower Complexity Bounds and Rate Optimal Algorithms
- Supervised Learning Under Distributed Features
- Quantized Consensus ADMM for Multi-Agent Distributed Optimization
- A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization
- Analysis of Distributed ADMM Algorithm for Consensus Optimization in Presence of Node Error
- GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning
- DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning
- Distributed Stochastic Consensus Optimization with Momentum for Nonconvex Nonsmooth Problems
- Privacy-preserving Incremental ADMM for Decentralized Consensus Optimization
- Communication-Censored Linearized ADMM for Decentralized Consensus Optimization
- NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization
- DC-DistADMM: ADMM Algorithm for Constrained Distributed Optimization over Directed Graphs
- Can Primal Methods Outperform Primal-dual Methods in Decentralized Dynamic Optimization?
- Decomposing Linearly Constrained Nonconvex Problems by a Proximal Primal Dual Approach: Algorithms, Convergence, and Applications
- Improved Convergence Rates for Distributed Resource Allocation
- Decentralized Resource Allocation via Dual Consensus ADMM
- Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function
- Decentralized Inexact Proximal Gradient Method With Network-Independent Stepsizes for Convex Composite Optimization
- Universal gradient descent
- Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solutions for Nonconvex Distributed Optimization
- Variance-Reduced Stochastic Quasi-Newton Methods for Decentralized Learning: Part I
- Linearized ADMM for Non-convex Non-smooth Optimization with Convergence Analysis
- Exact Diffusion for Distributed Optimization and Learning --- Part I: Algorithm Development
- A Decentralized Proximal Point-type Method for Saddle Point Problems
- Stochastic Proximal Gradient Consensus Over Random Networks
- Distributed ADMM with Synergetic Communication and Computation
- Implicit Tracking-Based Distributed Constraint-Coupled Optimization
- EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
- Distributed Resource Allocation for Epidemic control
- Fast and Robust Sparsity Learning over Networks: A Decentralized Surrogate Median Regression Approach
- On Nonconvex Decentralized Gradient Descent
- Fully Decentralized Federated Learning Based Beamforming Design for UAV Communications
- Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization
- Towards Plausible Differentially Private ADMM Based Distributed Machine Learning
- Federated Semi-Supervised Learning with Class Distribution Mismatch
- Decentralized Consensus Optimization with Asynchrony and Delays
- Robust and Communication-Efficient Collaborative Learning
- Decentralized Consensus Algorithm with Delayed and Stochastic Gradients
- Communication-Efficient Algorithms for Decentralized and Stochastic Optimization
- Decentralized Markov Chain Gradient Descent
- A Survey of Distributed Optimization Methods for Multi-Robot Systems
- A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraints
- Parallel Calibration for Sensor Array Radio Interferometers
- Decentralized Riemannian Gradient Descent on the Stiefel Manifold
- NEXT: In-Network Nonconvex Optimization
- Composite Optimization with Coupling Constraints via Dual Proximal Gradient Method with Applications to Asynchronous Networks
- Distributed Dual Gradient Tracking for Resource Allocation in Unbalanced Networks
- NPGA: A Unified Algorithmic Framework for Decentralized Constraint-Coupled Optimization
- Differentially Private Decentralized Optimization with Relay Communication
- Generalized Nash Equilibrium Problem by the Alternating Direction Method of Multipliers
- Local Differential Privacy in Decentralized Optimization
- A Bregman Splitting Algorithm for Distributed Optimization over Networks
- Bregman Parallel Direction Method of Multipliers for Distributed Optimization via Mirror Averaging
- Toward Model Parallelism for Deep Neural Network based on Gradient-free ADMM Framework
- Supervised MPC control of large-scale electricity networks via clustering methods
- Distributed Convex Optimization With Coupling Constraints Over Time-Varying Directed Graphs
- Learning-Accelerated ADMM for Distributed Optimal Power Flow
- A Fully Parallel Primal-Dual Algorithm for Centralized and Distributed Optimization
- Cloud-aided collaborative estimation by ADMM-RLS algorithms for connected vehicle prognostics
- First-order Methods with Convergence Rates for Multi-agent Systems on Semidefinite Matrix Spaces
- Consensus-based Distributed Discrete Optimal Transport for Decentralized Resource Matching
- Optimal Gradient Tracking for Decentralized Optimization
- On linear convergence of two decentralized algorithms
- Augmented Lagrangian Optimization under Fixed-Point Arithmetic
- Decentralized Non-Convex Learning with Linearly Coupled Constraints
- Decentralized and Equitable Optimal Transport
- Distributed Robust Subspace Recovery
- Zeroth-Order Feedback Optimization for Cooperative Multi-Agent Systems
- Distributed Control of Truss Robots Using Consensus Alternating Direction Method of Multipliers
- Distributed Partitioned Big-Data Optimization via Asynchronous Dual Decomposition
- Robust Calibration of Radio Interferometers in Multi-Frequency Scenario
- Distributed Dual Coordinate Ascent in General Tree Networks and Communication Network Effect on Synchronous Machine Learning
- Improving Rate of Convergence via Gain Adaptation in Multi-Agent Distributed ADMM Framework
- Controlling Power and Virtual Inertia from Storage for Frequency Response
- A Fast Proximal Gradient Algorithm for Decentralized Composite Optimization over Directed Networks