Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Function

Abstract

In this paper, we present a novel penalty model called ExPen for optimization over the Stiefel manifold. Different from existing penalty functions for orthogonality constraints, ExPen adopts a smooth penalty function without using any first-order derivative of the objective function. We show that all the first-order stationary points of ExPen with a sufficiently large penalty parameter are either feasible, namely, are the first-order stationary points of the original optimization problem, or far from the Stiefel manifold. Besides, the original problem and ExPen share the same second-order stationary points. Remarkably, the exact gradient and Hessian of ExPen are easy to compute. As a consequence, abundant algorithm resources in unconstrained optimization can be applied straightforwardly to solve ExPen.

revised version, 28 pages

References in corpus (5)

• Trivializations for Gradient-Based Optimization on Manifolds
• R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
• Escaping from saddle points on Riemannian manifolds
• Escape saddle points faster on manifolds via perturbed Riemannian stochastic recursive gradient
• Multipliers Correction Methods for Optimization Problems over the Stiefel Manifold

Cited by in corpus (1)

• Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function