Constrained composite optimization and augmented Lagrangian methods

Abstract

We investigate finite-dimensional constrained structured optimization problems, featuring composite objective functions and set-membership constraints. Offering an expressive yet simple language, this problem class provides a modeling framework for a variety of applications. We study stationarity and regularity concepts, and propose a flexible augmented Lagrangian scheme. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. It is demonstrated how the inner subproblems can be solved by off-the-shelf proximal methods, notwithstanding the possibility to adopt any solvers, insofar as they return approximate stationary points. Finally, we describe our matrix-free implementation of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained composite programs as a modeling tool and expose difficulties arising in this vast problem class.

41 pages, 5 figures

References in corpus (3)

• Proximal Gradient Algorithms under Local Lipschitz Gradient Continuity: A Convergence and Robustness Analysis of PANOC
• On implicit variables in optimization theory
• Neural Network Training as an Optimal Control Problem: An Augmented Lagrangian Approach

Cited by in corpus (6)

• Proximal Gradient Algorithms under Local Lipschitz Gradient Continuity: A Convergence and Robustness Analysis of PANOC
• Proximal gradient methods beyond monotony
• Implicit augmented Lagrangian and generalized optimization
• Local properties and augmented Lagrangians in fully nonconvex composite optimization
• Fuzzy multiplier, sum and intersection rules in non-Lipschitzian settings: decoupling approach revisited
• An interior proximal gradient method for nonconvex optimization