SPRING: A fast stochastic proximal alternating method for non-smooth non-convex optimization
arXiv:2002.12266
Abstract
We introduce SPRING, a novel stochastic proximal alternating linearized minimization algorithm for solving a class of non-smooth and non-convex optimization problems. Large-scale imaging problems are becoming increasingly prevalent due to advances in data acquisition and computational capabilities. Motivated by the success of stochastic optimization methods, we propose a stochastic variant of proximal alternating linearized minimization (PALM) algorithm \cite{bolte2014proximal}. We provide global convergence guarantees, demonstrating that our proposed method with variance-reduced stochastic gradient estimators, such as SAGA \cite{SAGA} and SARAH \cite{sarah}, achieves state-of-the-art oracle complexities. We also demonstrate the efficacy of our algorithm via several numerical examples including sparse non-negative matrix factorization, sparse principal component analysis, and blind image deconvolution.
arXiv comment28 pages, 11 page appendix
References in corpus (11)
- Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
- Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems
- Adversarial Regularizers in Inverse Problems
- The Practicality of Stochastic Optimization in Imaging Inverse Problems
- SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms
- The Sound of APALM Clapping: Faster Nonsmooth Nonconvex Optimization with Stochastic Asynchronous PALM
- A Proximal Stochastic Gradient Method with Progressive Variance Reduction
- The Asynchronous PALM Algorithm for Nonsmooth Nonconvex Problems
- Momentum Schemes with Stochastic Variance Reduction for Nonconvex Composite Optimization
- Lower Bounds for Smooth Nonconvex Finite-Sum Optimization