Constrained composite optimization and augmented Lagrangian methods

Abstract

We investigate finite-dimensional constrained structured optimization problems, featuring composite objective functions and set-membership constraints. Offering an expressive yet simple language, this problem class provides a modeling framework for a variety of applications. We study stationarity and regularity concepts, and propose a flexible augmented Lagrangian scheme. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. It is demonstrated how the inner subproblems can be solved by off-the-shelf proximal methods, notwithstanding the possibility to adopt any solvers, insofar as they return approximate stationary points. Finally, we describe our matrix-free implementation of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained composite programs as a modeling tool and expose difficulties arising in this vast problem class.

arXiv comment41 pages, 5 figures

References in corpus (10)

• Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone line-search algorithms
• The proximal augmented Lagrangian method for nonsmooth composite optimization
• Second-Order Switching Time Optimization for Switched Dynamical Systems
• An Augmented Lagrangian Method for Optimization Problems in Banach Spaces
• On the Complexity of an Augmented Lagrangian Method for Nonconvex Optimization
• Proximal Gradient Algorithms under Local Lipschitz Gradient Continuity: A Convergence and Robustness Analysis of PANOC
• Asymptotic stationarity and regularity for nonsmooth optimization problems
• A Sequential Homotopy Method for Mathematical Programming Problems
• On implicit variables in optimization theory
• Neural Network Training as an Optimal Control Problem: An Augmented Lagrangian Approach

Cited by in corpus (6)

• Proximal Gradient Algorithms under Local Lipschitz Gradient Continuity: A Convergence and Robustness Analysis of PANOC
• Proximal gradient methods beyond monotony
• Implicit augmented Lagrangian and generalized optimization
• Local properties and augmented Lagrangians in fully nonconvex composite optimization
• Fuzzy multiplier, sum and intersection rules in non-Lipschitzian settings: decoupling approach revisited
• An interior proximal gradient method for nonconvex optimization