SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization
arXiv:2207.08195 · doi:10.1007/s10589-023-00550-8
Abstract
We introduce SPIRAL, a SuPerlinearly convergent Incremental pRoximal ALgorithm, for solving nonconvex regularized finite sum problems under a relative smoothness assumption. Each iteration of SPIRAL consists of an inner and an outer loop. It combines incremental gradient updates with a linesearch that has the remarkable property of never being triggered asymptotically, leading to superlinear convergence under mild assumptions at the limit point. Simulation results with L-BFGS directions on different convex, nonconvex, and non-Lipschitz differentiable problems show that our algorithm, as well as its adaptive variant, are competitive to the state of the art.
References in corpus (11)
- Phase Retrieval via Wirtinger Flow: Theory and Algorithms
- A Geometric Analysis of Phase Retrieval
- Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone line-search algorithms
- Why Random Reshuffling Beats Stochastic Gradient Descent
- A Bregman forward-backward linesearch algorithm for nonconvex composite optimization: superlinear convergence to nonisolated local minima
- SuperMann: a superlinearly convergent algorithm for finding fixed points of nonexpansive operators
- On the acceleration of forward-backward splitting via an inexact Newton method
- Proximal Gradient Algorithms under Local Lipschitz Gradient Continuity: A Convergence and Robustness Analysis of PANOC
- Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems
- Variance-Reduced Stochastic Quasi-Newton Methods for Decentralized Learning: Part I
- Bregman Finito/MISO for nonconvex regularized finite sum minimization without Lipschitz gradient continuity