Generalized Left-Localized Cayley Parametrization for Optimization with Orthogonality Constraints

Abstract

We present a reformulation of optimization problems over the Stiefel manifold by using a Cayley-type transform, named the generalized left-localized Cayley transform, for the Stiefel manifold. The reformulated optimization problem is defined over a vector space, whereby we can apply directly powerful computational arts designed for optimization over a vector space. The proposed Cayley-type transform enjoys several key properties which are useful to (i) study relations between the original problem and the proposed problem; (ii) check the conditions to guarantee the global convergence of optimization algorithms. Numerical experiments demonstrate that the proposed algorithm outperforms the standard algorithms designed with a retraction on the Stiefel manifold.

arXiv comment44 pages

References in corpus (10)

• Manopt, a Matlab toolbox for optimization on manifolds
• Generalized power method for sparse principal component analysis
• Stochastic Variance Reduction for Nonconvex Optimization
• AdaGrad stepsizes: Sharp convergence over nonconvex landscapes
• A new, globally convergent Riemannian conjugate gradient method
• On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
• Natasha 2: Faster Non-Convex Optimization Than SGD
• A Framework for Generalising the Newton Method and Other Iterative Methods from Euclidean Space to Manifolds
• Accelerated Optimization With Orthogonality Constraints
• Riemannian Newton-type methods for joint diagonalization on the Stiefel manifold with application to independent component analysis

Cited by in corpus (1)

• A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering