3k citations · 9k across the 154 of their papers we have counts for
87 papers · 1 filter
Det-CGD: Compressed Gradient Descent with Matrix Stepsizes for Non-Convex Optimization
Hanmin Li, Avetik Karagulyan, Peter Richtárik
This paper introduces a new method for minimizing matrix-smooth non-convex objectives through the use of novel Compressed Gradient Descent (CGD) algorithms enhanced with a matrix-v…
Optimal Time Complexities of Parallel Stochastic Optimization Methods Under a Fixed Computation Model
Alexander Tyurin, Peter Richtárik
Parallelization is a popular strategy for improving the performance of iterative algorithms. Optimization methods are no exception: design of efficient parallel optimization method…
High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance
Abdurakhmon Sadiev, Marina Danilova, Eduard Gorbunov +5
During recent years the interest of optimization and machine learning communities in high-probability convergence of stochastic optimization methods has been growing. One of the ma…
Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes
Konstantin Mishchenko, Slavomír Hanzely, Peter Richtárik
In this work, we consider the problem of minimizing the sum of Moreau envelopes of given functions, which has previously appeared in the context of meta-learning and personalized f…
A Damped Newton Method Achieves Global and Local Quadratic Convergence Rate
Slavomír Hanzely, Dmitry Kamzolov, Dmitry Pasechnyuk +3
In this paper, we present the first stepsize schedule for Newton method resulting in fast global and local convergence guarantees. In particular, a) we prove an $O\left( \frac 1 {k…
Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization
Soumia Boucherouite, Grigory Malinovsky, Peter Richtárik +1
In this paper, we propose a new zero order optimization method called minibatch stochastic three points (MiSTP) method to solve an unconstrained minimization problem in a setting w…