Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Function
arXiv:2110.08986
Abstract
In this paper, we present a novel penalty model called ExPen for optimization over the Stiefel manifold. Different from existing penalty functions for orthogonality constraints, ExPen adopts a smooth penalty function without using any first-order derivative of the objective function. We show that all the first-order stationary points of ExPen with a sufficiently large penalty parameter are either feasible, namely, are the first-order stationary points of the original optimization problem, or far from the Stiefel manifold. Besides, the original problem and ExPen share the same second-order stationary points. Remarkably, the exact gradient and Hessian of ExPen are easy to compute. As a consequence, abundant algorithm resources in unconstrained optimization can be applied straightforwardly to solve ExPen.
arXiv commentrevised version, 28 pages
References in corpus (13)
- Riemannian Adaptive Optimization Methods
- A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions
- Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group
- Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints
- Trivializations for Gradient-Based Optimization on Manifolds
- Efficiently escaping saddle points on manifolds
- Towards Riemannian Accelerated Gradient Methods
- R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
- Accelerated Optimization With Orthogonality Constraints
- Escaping from saddle points on Riemannian manifolds
- Cubic Regularization with Momentum for Nonconvex Optimization
- Escape saddle points faster on manifolds via perturbed Riemannian stochastic recursive gradient
- Multipliers Correction Methods for Optimization Problems over the Stiefel Manifold