15 citations · 78 across the 24 of their papers we have counts for
19 papers · 1 filter
A Unified Primal-Dual Recipe for Accelerating Three-Operator Splitting Methods
Abdurakhmon Sadiev, Laurent Condat, Peter Richtárik
Composite optimization problems, formulated as the minimization of three functions, are ubiquitous in large-scale machine learning and signal processing. While state-of-the-art spl…
A Nesterov-Accelerated Primal-Dual Splitting Algorithm for Convex Nonsmooth Optimization
Laurent Condat, Abdurakhmon Sadiev, Peter Richtárik
We investigate the integration of Nesterov-type acceleration into primal-dual methods for structured convex optimization. While proximal splitting algorithms efficiently handle com…
Better LMO-based Momentum Methods with Second-Order Information
Sarit Khirirat, Abdurakhmon Sadiev, Yury Demidovich +1
The use of momentum in stochastic optimization algorithms has shown empirical success across a range of machine learning tasks. Recently, a new class of stochastic momentum algorit…
Tight Lower Bounds and Optimal Algorithms for Stochastic Nonconvex Optimization with Heavy-Tailed Noise
Adrien Fradin, Abdurakhmon Sadiev, Laurent Condat +1
We study stochastic nonconvex optimization under heavy-tailed noise. In this setting, the stochastic gradients only have bounded -th central moment (-BCM) for some $p \in (1,…
Improved Convergence in Parameter-Agnostic Error Feedback through Momentum
Abdurakhmon Sadiev, Yury Demidovich, Igor Sokolov +3
Communication compression is essential for scalable distributed training of modern machine learning models, but it often degrades convergence due to the noise it introduces. Error…
Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits
Abdurakhmon Sadiev, Peter Richtárik, Ilyas Fatkhullin
Heavy-tailed noise is pervasive in modern machine learning applications, arising from data heterogeneity, outliers, and non-stationary stochastic environments. While second-order m…
Differentially Private Random Block Coordinate Descent
Artavazd Maranjyan, Abdurakhmon Sadiev, Peter Richtárik
Coordinate Descent (CD) methods have gained significant attention in machine learning due to their effectiveness in solving high-dimensional problems and their ability to decompose…
SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
Avetik Karagulyan, Egor Shulgin, Abdurakhmon Sadiev +1
Cross-device training is a crucial subfield of federated learning, where the number of clients can reach into the billions. Standard approaches and local methods are prone to issue…
A Unified Theory of Stochastic Proximal Point Methods without Smoothness
Peter Richtárik, Abdurakhmon Sadiev, Yury Demidovich
This paper presents a comprehensive analysis of a broad range of variations of the stochastic proximal point method (SPPM). Proximal point methods have attracted considerable inter…
Stochastic Proximal Point Methods for Monotone Inclusions under Expected Similarity
Abdurakhmon Sadiev, Laurent Condat, Peter Richtárik
Monotone inclusions have a wide range of applications, including minimization, saddle-point, and equilibria problems. We introduce new stochastic algorithms, with or without varian…
High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
Eduard Gorbunov, Abdurakhmon Sadiev, Marina Danilova +5
High-probability analysis of stochastic first-order optimization methods under mild assumptions on the noise has been gaining a lot of attention in recent years. Typically, gradien…
High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance
Abdurakhmon Sadiev, Marina Danilova, Eduard Gorbunov +5
During recent years the interest of optimization and machine learning communities in high-probability convergence of stochastic optimization methods has been growing. One of the ma…
Stochastic Gradient Methods with Preconditioned Updates
Abdurakhmon Sadiev, Aleksandr Beznosikov, Abdulla Jasem Almansoori +3
This work considers the non-convex finite sum minimization problem. There are several algorithms for such problems, but existing methods often work poorly when the problem is badly…
An Approach for Non-Convex Uniformly Concave Structured Saddle Point Problem
Mohammad Alkousa, Alexander Gasnikov, Pavel Dvurechensky +2
Recently, saddle point problems have received much attention due to their powerful modeling capability for a lot of problems from diverse domains. Applications of these problems oc…
Optimal Algorithms for Decentralized Stochastic Variational Inequalities
Dmitry Kovalev, Aleksandr Beznosikov, Abdurakhmon Sadiev +3
Variational inequalities are a formalism that includes games, minimization, saddle point, and equilibrium problems as special cases. Methods for variational inequalities are theref…
Decentralized Personalized Federated Learning: Lower Bounds and Optimal Algorithm for All Personalization Modes
Abdurakhmon Sadiev, Ekaterina Borodich, Aleksandr Beznosikov +5
This paper considers the problem of decentralized, personalized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the…
Solving smooth min-min and min-max problems by mixed oracle algorithms
Egor Gladin, Abdurakhmon Sadiev, Alexander Gasnikov +3
In this paper, we consider two types of problems that have some similarity in their structure, namely, min-min problems and min-max saddle-point problems. Our approach is based on…
Zeroth-Order Algorithms for Smooth Saddle-Point Problems
Abdurakhmon Sadiev, Aleksandr Beznosikov, Pavel Dvurechensky +1
Saddle-point problems have recently gained increased attention from the machine learning community, mainly due to applications in training Generative Adversarial Networks using sto…
Gradient-Free Methods for Saddle-Point Problem
Aleksandr Beznosikov, Abdurakhmon Sadiev, Alexander Gasnikov
In the paper, we generalize the approach Gasnikov et. al, 2017, which allows to solve (stochastic) convex optimization problems with an inexact gradient-free oracle, to the convex-…