Universal gradient descent
arXiv:1711.00394
Abstract
In this book we collect many different and useful facts around gradient descent method. First of all we consider gradient descent with inexact oracle. We build a general model of optimized function that include composite optimization approach, level's methods, proximal methods etc. Then we investigate primal-dual properties of the gradient descent in general model set-up. At the end we generalize method to universal one.
MIPT, 291 pages (in Russian)
References in corpus (8)
- Convex Optimization for Big Data
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
- Qualitatively characterizing neural network optimization problems
- Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm
- How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD
- The proximal point method revisited
- An Accelerated Directional Derivative Method for Smooth Stochastic Convex Optimization
- Learning From An Optimization Viewpoint
Cited by in corpus (5)
- Recent theoretical advances in decentralized distributed convex optimization
- Adaptive Gradient Descent for Convex and Non-Convex Stochastic Optimization
- Near-Optimal Hyperfast Second-Order Method for convex optimization and its Sliding
- Method with Batching for Stochastic Finite-Sum Variational Inequalities in Non-Euclidean Setting
- Parallel and Distributed algorithms for ML problems