Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold

Abstract

We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objective function and thus tend to converge slowly in practice. Algorithms in the second class are proximal point algorithms, which involve subproblems that can be as difficult as the original problem. Algorithms in the third class are based on operator-splitting techniques, but they usually lack rigorous convergence guarantees. In this paper, we propose a retraction-based proximal gradient method for solving this class of problems. We prove that the proposed method globally converges to a stationary point. Iteration complexity for obtaining an $\epsilon$-stationary solution is also analyzed. Numerical results on solving sparse PCA and compressed modes problems are reported to demonstrate the advantages of the proposed method.

References in corpus (12)

• Nearly unbiased variable selection under minimax concave penalty
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• Manopt, a Matlab toolbox for optimization on manifolds
• Generalized power method for sparse principal component analysis
• Global rates of convergence for nonconvex optimization on manifolds
• Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture
• Compressed Modes for Variational Problems in Mathematics and Physics
• Nonconvex phase synchronization
• A Second Order Non-Smooth Variational Model for Restoring Manifold-Valued Images
• Convergence of inexact descent methods for nonconvex optimization on Riemannian manifolds
• Vector Transport-Free SVRG with General Retraction for Riemannian Optimization: Complexity Analysis and Practical Implementation
• Primal-Dual Optimization Algorithms over Riemannian Manifolds: an Iteration Complexity Analysis

Cited by in corpus (1)

• Multi-Rank Sparse and Functional PCA: Manifold Optimization and Iterative Deflation Techniques