Understanding Notions of Stationarity in Non-Smooth Optimization
arXiv:2006.14901
Abstract
Many contemporary applications in signal processing and machine learning give rise to structured non-convex non-smooth optimization problems that can often be tackled by simple iterative methods quite effectively. One of the keys to understanding such a phenomenon---and, in fact, one of the very difficult conundrums even for experts---lie in the study of "stationary points" of the problem in question. Unlike smooth optimization, for which the definition of a stationary point is rather standard, there is a myriad of definitions of stationarity in non-smooth optimization. In this article, we give an introduction to different stationarity concepts for several important classes of non-convex non-smooth functions and discuss the geometric interpretations and further clarify the relationship among these different concepts. We then demonstrate the relevance of these constructions in some representative applications and how they could affect the performance of iterative methods for tackling these applications.
arXiv commentAccepted for publication in IEEE Signal Processing Magazine, 2020
References in corpus (13)
- Nearly unbiased variable selection under minimax concave penalty
- High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
- An overview of low-rank matrix recovery from incomplete observations
- Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture
- Matrix Completion has No Spurious Local Minimum
- Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method
- Phase Retrieval with Application to Optical Imaging
- Nonconvex Sparse Logistic Regression with Weakly Convex Regularization
- Composite optimization for robust blind deconvolution
- Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods
- Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates
- On the Finite Number of Directional Stationary Values of Piecewise Programs
- On the Global Minimizers of Real Robust Phase Retrieval with Sparse Noise