66 citations · 178 across the 17 of their papers we have counts for
21 papers
On the Spectral Dependence of Decentralized Stochastic Gradient Tracking
Sulaiman A. Alghunaim, Haoyuan Cai, Ali H. Sayed
We study how the spectral geometry of the mixing matrix enters the convergence of decentralized stochastic gradient tracking (GT) for smooth strongly convex problems. General bound…
CED-EF: Compressed Exact Diffusion with Error Feedback for Multi-Agent Learning
Sulaiman A. Alghunaim, Kun Yuan
We study decentralized stochastic optimization over a network of agents under compressed communication. We propose CED-EF, an exact diffusion-based method with error feedback t…
Stochastic Gradient Tracking over Time-Varying Networks: One-Step Lyapunov Analysis
Sulaiman A. Alghunaim
We study decentralized stochastic gradient tracking over a time-varying network of agents under a uniform window-mixing condition. Products of consecutive doubly stochastic…
DAMA: A Unified Accelerated Approach for Decentralized Nonconvex Minimax Optimization-Part II: Convergence and Performance Analyses
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
In Part I of this work [1], we developed an accelerated algorithmic framework, DAMA (Decentralized Accelerated Minimax Approach), for nonconvex Polyak-Lojasiewicz (PL) minimax opti…
DAMA: A Unified Accelerated Approach for Decentralized Nonconvex Minimax Optimization-Part I: Algorithm Development and Results
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
In this work and its accompanying Part II [1], we develop an accelerated algorithmic framework, DAMA (Decentralized Accelerated Minimax Approach), for nonconvex Polyak-Lojasiewicz…
Communication-Efficient Decentralized Stochastic Minimax Optimization
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
In this work, we study decentralized stochastic nonconvex Polyak--Łojasiewicz minimax problems and propose a communication-efficient algorithm. Motivated by the efficiency of local…
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least …
Diffusion Stochastic Optimization for Min-Max Problems
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
The optimistic gradient method is useful in addressing minimax optimization problems. Motivated by the observation that the conventional stochastic version suffers from the need fo…
Achieving Linear Speedup with ProxSkip in Distributed Stochastic Optimization
Luyao Guo, Sulaiman A. Alghunaim, Kun Yuan +2
The ProxSkip algorithm for distributed optimization is gaining increasing attention due to its effectiveness in reducing communication. However, existing analyses of ProxSkip are l…
Local Exact-Diffusion for Decentralized Optimization and Learning
Sulaiman A. Alghunaim
Distributed optimization methods with local updates have recently attracted a lot of attention due to their potential to reduce the communication cost of distributed methods. In th…
An Enhanced Gradient-Tracking Bound for Distributed Online Stochastic Convex Optimization
Sulaiman A. Alghunaim, Kun Yuan
Gradient-tracking (GT) based decentralized methods have emerged as an effective and viable alternative method to decentralized (stochastic) gradient descent (DSGD) when solving dis…
On the Performance of Gradient Tracking with Local Updates
Edward Duc Hien Nguyen, Sulaiman A. Alghunaim, Kun Yuan +1
We study the decentralized optimization problem where a network of agents seeks to minimize the average of a set of heterogeneous non-convex cost functions distributedly. State…
A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning
Sulaiman A. Alghunaim, Kun Yuan
We study the consensus decentralized optimization problem where the objective function is the average of agents private non-convex cost functions; moreover, the agents can only…
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
Kun Yuan, Sulaiman A. Alghunaim, Xinmeng Huang
We consider the decentralized stochastic optimization problems, where a network of nodes, each owning a local cost function, cooperate to find a minimizer of the globally-avera…
A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features
Sulaiman A. Alghunaim, Ming Yan, Ali H. Sayed
This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function couplin…
Decentralized Proximal Gradient Algorithms with Linear Convergence Rates
Sulaiman A. Alghunaim, Ernest K. Ryu, Kun Yuan +1
This work studies a class of non-smooth decentralized multi-agent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a c…
A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization
Sulaiman A. Alghunaim, Kun Yuan, Ali H. Sayed
Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with n…
Linear Convergence of Primal-Dual Gradient Methods and their Performance in Distributed Optimization
Sulaiman A. Alghunaim, Ali H. Sayed
In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problem…
On the Influence of Bias-Correction on Distributed Stochastic Optimization
Kun Yuan, Sulaiman A. Alghunaim, Bicheng Ying +1
Various bias-correction methods such as EXTRA, gradient tracking methods, and exact diffusion have been proposed recently to solve distributed {\em deterministic} optimization prob…
A Proximal Diffusion Strategy for Multi-Agent Optimization with Sparse Affine Constraints
Sulaiman A. Alghunaim, Kun Yuan, Ali H. Sayed
This work develops a proximal primal-dual decentralized strategy for multi-agent optimization problems that involve multiple coupled affine constraints, where each constraint may i…
Distributed Coupled Multi-Agent Stochastic Optimization
Sulaiman A. Alghunaim, Ali H. Sayed
This work develops effective distributed strategies for the solution of constrained multi-agent stochastic optimization problems with coupled parameters across the agents. In this…