Heuristic adaptive fast gradient method in stochastic optimization tasks

Abstract

In this paper, we present a heuristic adaptive fast gradient method. We show that in practice our method has a better convergence rate than popular today optimization methods. Moreover, we justify our method and point out some problems that do not allow us to obtain theoretical estimates.

arXiv commentin Russian

References in corpus (5)

• Large Deviations of Vector-valued Martingales in 2-Smooth Normed Spaces
• A Universal Algorithm for Variational Inequalities Adaptive to Smoothness and Noise
• Online Adaptive Methods, Universality and Acceleration
• Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
• Optimal Adaptive and Accelerated Stochastic Gradient Descent