3k citations · 9k across the 154 of their papers we have counts for
171 papers
Understanding Progressive Training Through the Framework of Randomized Coordinate Descent
Rafał Szlendak, Elnur Gasanov, Peter Richtárik
We propose a Randomized Progressive Training algorithm (RPT) -- a stochastic proxy for the well-known Progressive Training method (PT) (Karras et al., 2017). Originally designed to…
A Guide Through the Zoo of Biased SGD
Yury Demidovich, Grigory Malinovsky, Igor Sokolov +1
Stochastic Gradient Descent (SGD) is arguably the most important single algorithm in modern machine learning. Although SGD with unbiased gradient estimators has been studied extens…
Det-CGD: Compressed Gradient Descent with Matrix Stepsizes for Non-Convex Optimization
Hanmin Li, Avetik Karagulyan, Peter Richtárik
This paper introduces a new method for minimizing matrix-smooth non-convex objectives through the use of novel Compressed Gradient Descent (CGD) algorithms enhanced with a matrix-v…
Optimal Time Complexities of Parallel Stochastic Optimization Methods Under a Fixed Computation Model
Alexander Tyurin, Peter Richtárik
Parallelization is a popular strategy for improving the performance of iterative algorithms. Optimization methods are no exception: design of efficient parallel optimization method…
High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance
Abdurakhmon Sadiev, Marina Danilova, Eduard Gorbunov +5
During recent years the interest of optimization and machine learning communities in high-probability convergence of stochastic optimization methods has been growing. One of the ma…
Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes
Konstantin Mishchenko, Slavomír Hanzely, Peter Richtárik
In this work, we consider the problem of minimizing the sum of Moreau envelopes of given functions, which has previously appeared in the context of meta-learning and personalized f…