10 citations · 26 across the 11 of their papers we have counts for
11 papers
The Stochastic Multi-Proximal Method for Nonsmooth Optimization
Laurent Condat, Elnur Gasanov, Peter Richtárik
Stochastic gradient descent type methods are ubiquitous in machine learning, but they are only applicable to the optimization of differentiable functions. Proximal algorithms are m…
Speeding up Stochastic Proximal Optimization in the High Hessian Dissimilarity Setting
Elnur Gasanov, Peter Richtárik
Stochastic proximal point methods have recently garnered renewed attention within the optimization community, primarily due to their desirable theoretical properties. Notably, thes…
Error Feedback Reloaded: From Quadratic to Arithmetic Mean of Smoothness Constants
Peter Richtárik, Elnur Gasanov, Konstantin Burlachenko
Error Feedback (EF) is a highly popular and immensely effective mechanism for fixing convergence issues which arise in distributed training methods (such as distributed GD or SGD)…
Understanding Progressive Training Through the Framework of Randomized Coordinate Descent
Rafał Szlendak, Elnur Gasanov, Peter Richtárik
We propose a Randomized Progressive Training algorithm (RPT) -- a stochastic proxy for the well-known Progressive Training method (PT) (Karras et al., 2017). Originally designed to…
Error Feedback Shines when Features are Rare
Peter Richtárik, Elnur Gasanov, Konstantin Burlachenko
We provide the first proof that gradient descent with greedy sparsification and error feedback $\left({\…
Adaptive Compression for Communication-Efficient Distributed Training
Maksim Makarenko, Elnur Gasanov, Rustem Islamov +2
We propose Adaptive Compressed Gradient Descent (AdaCGD) - a novel optimization algorithm for communication-efficient training of supervised machine learning models with adaptive c…
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation
Peter Richtárik, Igor Sokolov, Ilyas Fatkhullin +3
We propose and study a new class of gradient communication mechanisms for communication-efficient training -- three point compressors (3PC) -- as well as efficient distributed nonc…
FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning
Elnur Gasanov, Ahmed Khaled, Samuel Horváth +1
Federated Learning (FL) is an increasingly popular machine learning paradigm in which multiple nodes try to collaboratively learn under privacy, communication and multiple heteroge…
Lower Bounds and Optimal Algorithms for Smooth and Strongly Convex Decentralized Optimization Over Time-Varying Networks
Dmitry Kovalev, Elnur Gasanov, Peter Richtárik +1
We consider the task of minimizing the sum of smooth and strongly convex functions stored in a decentralized manner across the nodes of a communication network whose links are allo…
From Local SGD to Local Fixed-Point Methods for Federated Learning
Grigory Malinovsky, Dmitry Kovalev, Elnur Gasanov +2
Most algorithms for solving optimization problems or finding saddle points of convex-concave functions are fixed-point algorithms. In this work we consider the generic problem of f…
Stochastic Spectral and Conjugate Descent Methods
Dmitry Kovalev, Eduard Gorbunov, Elnur Gasanov +1
The state-of-the-art methods for solving optimization problems in big dimensions are variants of randomized coordinate descent (RCD). In this paper we introduce a fundamentally new…