The proximal point method revisited

Abstract

In this short survey, I revisit the role of the proximal point method in large scale optimization. I focus on three recent examples: a proximally guided subgradient method for weakly convex stochastic approximation, the prox-linear algorithm for minimizing compositions of convex functions and smooth maps, and Catalyst generic acceleration for regularized Empirical Risk Minimization.

11 pages, submitted to SIAG/OPT Views and News

References in corpus (12)

• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
• Proximal Stochastic Dual Coordinate Ascent
• On the low-rank approach for semidefinite programs arising in synchronization and community detection
• Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
• Tight Complexity Bounds for Optimizing Composite Objectives
• An optimal randomized incremental gradient method
• "Convex Until Proven Guilty": Dimension-Free Acceleration of Gradient Descent on Non-Convex Functions
• Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrieval
• Catalyst Acceleration for Gradient-Based Non-Convex Optimization
• Stochastic Methods for Composite and Weakly Convex Optimization Problems
• SMART: The Stochastic Monotone Aggregated Root-Finding Algorithm

Cited by in corpus (14)

• The importance of better models in stochastic optimization
• Stochastic subgradient method converges at the rate $O(k^{-1/4})$ on weakly convex functions
• On the Convergence Rate of Stochastic Mirror Descent for Nonsmooth Nonconvex Optimization
• Universal gradient descent
• Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods
• Hybrid Inexact BCD for Coupled Structured Matrix Factorization in Hyperspectral Super-Resolution
• Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learning
• A Unified Analysis of First-Order Methods for Smooth Games via Integral Quadratic Constraints
• Stochastic Variance-Reduced Prox-Linear Algorithms for Nonconvex Composite Optimization
• High-Dimensional Robust Mean Estimation via Gradient Descent
• Algorithms for Difference-of-Convex (DC) Programs Based on Difference-of-Moreau-Envelopes Smoothing
• On global properties of lower semicontinuous quadratically minorized functions
• A Stochastic Objective-Function-Free Adaptive Regularization Method with Optimal Complexity
• Stochastic Bias-Reduced Gradient Methods