ADMM and Accelerated ADMM as Continuous Dynamical Systems

Abstract

Recently, there has been an increasing interest in using tools from dynamical systems to analyze the behavior of simple optimization algorithms such as gradient descent and accelerated variants. This paper strengthens such connections by deriving the differential equations that model the continuous limit of the sequence of iterates generated by the alternating direction method of multipliers, as well as an accelerated variant. We employ the direct method of Lyapunov to analyze the stability of critical points of the dynamical systems and to obtain associated convergence rates.

References in corpus (4)

• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• A Variational Perspective on Accelerated Methods in Optimization
• First-order Methods Almost Always Avoid Saddle Points
• How is Distributed ADMM Affected by Network Topology?

Cited by in corpus (16)

• Hamiltonian Descent Methods
• A Nonsmooth Dynamical Systems Perspective on Accelerated Extensions of ADMM
• An Accelerated Correlation Filter Tracker
• Conformal Symplectic and Relativistic Optimization
• Gradient flows and proximal splitting methods: A unified view on accelerated and stochastic optimization
• Geometry of First-Order Methods and Adaptive Acceleration
• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
• Accelerating two projection methods via perturbations with application to Intensity-Modulated Radiation Therapy
• Finite-Time Convergence of Continuous-Time Optimization Algorithms via Differential Inclusions
• Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization
• ADMM-based Distributed State Estimation for Power Systems: Evaluation of Performance
• Learning-Accelerated ADMM for Distributed Optimal Power Flow
• A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate
• A Dynamical Systems Approach for Convergence of the Bayesian EM Algorithm
• On the Asymptotic Linear Convergence Speed of Anderson Acceleration Applied to ADMM
• DessiLBI: Exploring Structural Sparsity of Deep Networks via Differential Inclusion Paths