Universal gradient descent
arXiv:1711.00394
Abstract
In this book we collect many different and useful facts around gradient descent method. First of all we consider gradient descent with inexact oracle. We build a general model of optimized function that include composite optimization approach, level's methods, proximal methods etc. Then we investigate primal-dual properties of the gradient descent in general model set-up. At the end we generalize method to universal one.
arXiv commentMIPT, 291 pages (in Russian)
References in corpus (19)
- Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- Multi-Agent Distributed Optimization via Inexact Consensus ADMM
- Convex Optimization for Big Data
- Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
- Qualitatively characterizing neural network optimization problems
- A Universal Catalyst for First-Order Optimization
- Optimized first-order methods for smooth convex minimization
- An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback
- Optimization with First-Order Surrogate Functions
- Exact Worst-case Performance of First-order Methods for Composite Convex Optimization
- Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm
- Beyond Convexity: Stochastic Quasi-Convex Optimization
- How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD
- The proximal point method revisited
- An Accelerated Directional Derivative Method for Smooth Stochastic Convex Optimization
- Learning From An Optimization Viewpoint
- Stochastic Intermediate Gradient Method for Convex Problems with Inexact Stochastic Oracle
Cited by in corpus (5)
- Recent theoretical advances in decentralized distributed convex optimization
- Adaptive Gradient Descent for Convex and Non-Convex Stochastic Optimization
- Near-Optimal Hyperfast Second-Order Method for convex optimization and its Sliding
- Method with Batching for Stochastic Finite-Sum Variational Inequalities in Non-Euclidean Setting
- Parallel and Distributed algorithms for ML problems