Separable Approximations and Decomposition Methods for the Augmented Lagrangian

Abstract

In this paper we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian. In particular, we study and compare the Diagonal Quadratic Approximation Method (DQAM) of Mulvey and Ruszczyński and the Parallel Coordinate Descent Method (PCDM) of Richtárik and Takáč. We show that the two methods are equivalent for feasibility problems up to the selection of a single step-size parameter. Furthermore, we prove an improved complexity bound for PCDM under strong convexity, and show that this bound is at least $8(L^{\prime }/\bar{L})(\omega -1{)}^2$ times better than the best known bound for DQAM, where $\omega$ is the degree of partial separability and $L^{\prime }$ and $\bar{L}$ are the maximum and average of the block Lipschitz constants of the gradient of the quadratic penalty appearing in the augmented Lagrangian.

28 pages, 6 algorithms, 2 figures

References in corpus (8)

• Parallel Coordinate Descent for L1-Regularized Loss Minimization
• Parallel Coordinate Descent Methods for Big Data Optimization
• Inexact Coordinate Descent: Complexity and Preconditioning
• On the Complexity Analysis of Randomized Block-Coordinate Descent Methods
• Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function
• Parallel coordinate descent for the Adaboost problem
• A Randomized Nonmonotone Block Proximal Gradient Method for a Class of Structured Nonlinear Programming
• A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints

Cited by in corpus (1)

• Parallel Direction Method of Multipliers