Stochastic Methods for Composite and Weakly Convex Optimization Problems

Abstract

We consider minimization of stochastic functionals that are compositions of a (potentially) non-smooth convex function $h$ and smooth function $c$ and, more generally, stochastic weakly-convex functionals. We develop a family of stochastic methods---including a stochastic prox-linear algorithm and a stochastic (generalized) sub-gradient procedure---and prove that, under mild technical conditions, each converges to first-order stationary points of the stochastic objective. We provide experiments further investigating our methods on non-smooth phase retrieval problems; the experiments indicate the practical effectiveness of the procedures.

Cited by in corpus (9)

• Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
• The proximal point method revisited
• VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning
• Solving Almost all Systems of Random Quadratic Equations
• Robust and Scalable Power System State Estimation via Composite Optimization
• Strong Metric (Sub)regularity of KKT Mappings for Piecewise Linear-Quadratic Convex-Composite Optimization
• Larger is Better: The Effect of Learning Rates Enjoyed by Stochastic Optimization with Progressive Variance Reduction
• Robust Learning of Trimmed Estimators via Manifold Sampling
• Robust Implicit Backpropagation