Adaptive Stochastic Optimization
arXiv:2001.06699
Abstract
Optimization lies at the heart of machine learning and signal processing. Contemporary approaches based on the stochastic gradient method are non-adaptive in the sense that their implementation employs prescribed parameter values that need to be tuned for each application. This article summarizes recent research and motivates future work on adaptive stochastic optimization methods, which have the potential to offer significant computational savings when training large-scale systems.
References in corpus (4)
- SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
- SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient
- A Stochastic Line Search Method with Convergence Rate Analysis
- Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
Cited by in corpus (3)
- Stochastic Trust Region Methods with Trust Region Radius Depending on Probabilistic Models
- Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates
- Adaptive Regularization for Nonconvex Optimization Using Inexact Function Values and Randomly Perturbed Derivatives