66 citations · 178 across the 17 of their papers we have counts for
16 papers · 1 filter
On the Spectral Dependence of Decentralized Stochastic Gradient Tracking
Sulaiman A. Alghunaim, Haoyuan Cai, Ali H. Sayed
We study how the spectral geometry of the mixing matrix enters the convergence of decentralized stochastic gradient tracking (GT) for smooth strongly convex problems. General bound…
CED-EF: Compressed Exact Diffusion with Error Feedback for Multi-Agent Learning
Sulaiman A. Alghunaim, Kun Yuan
We study decentralized stochastic optimization over a network of agents under compressed communication. We propose CED-EF, an exact diffusion-based method with error feedback t…
Stochastic Gradient Tracking over Time-Varying Networks: One-Step Lyapunov Analysis
Sulaiman A. Alghunaim
We study decentralized stochastic gradient tracking over a time-varying network of agents under a uniform window-mixing condition. Products of consecutive doubly stochastic…
DAMA: A Unified Accelerated Approach for Decentralized Nonconvex Minimax Optimization-Part II: Convergence and Performance Analyses
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
In Part I of this work [1], we developed an accelerated algorithmic framework, DAMA (Decentralized Accelerated Minimax Approach), for nonconvex Polyak-Lojasiewicz (PL) minimax opti…
DAMA: A Unified Accelerated Approach for Decentralized Nonconvex Minimax Optimization-Part I: Algorithm Development and Results
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
In this work and its accompanying Part II [1], we develop an accelerated algorithmic framework, DAMA (Decentralized Accelerated Minimax Approach), for nonconvex Polyak-Lojasiewicz…
Communication-Efficient Decentralized Stochastic Minimax Optimization
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
In this work, we study decentralized stochastic nonconvex Polyak--Łojasiewicz minimax problems and propose a communication-efficient algorithm. Motivated by the efficiency of local…
Local Exact-Diffusion for Decentralized Optimization and Learning
Sulaiman A. Alghunaim
Distributed optimization methods with local updates have recently attracted a lot of attention due to their potential to reduce the communication cost of distributed methods. In th…
An Enhanced Gradient-Tracking Bound for Distributed Online Stochastic Convex Optimization
Sulaiman A. Alghunaim, Kun Yuan
Gradient-tracking (GT) based decentralized methods have emerged as an effective and viable alternative method to decentralized (stochastic) gradient descent (DSGD) when solving dis…
On the Performance of Gradient Tracking with Local Updates
Edward Duc Hien Nguyen, Sulaiman A. Alghunaim, Kun Yuan +1
We study the decentralized optimization problem where a network of agents seeks to minimize the average of a set of heterogeneous non-convex cost functions distributedly. State…
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
Kun Yuan, Sulaiman A. Alghunaim, Xinmeng Huang
We consider the decentralized stochastic optimization problems, where a network of nodes, each owning a local cost function, cooperate to find a minimizer of the globally-avera…
A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features
Sulaiman A. Alghunaim, Ming Yan, Ali H. Sayed
This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function couplin…
Decentralized Proximal Gradient Algorithms with Linear Convergence Rates
Sulaiman A. Alghunaim, Ernest K. Ryu, Kun Yuan +1
This work studies a class of non-smooth decentralized multi-agent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a c…
A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization
Sulaiman A. Alghunaim, Kun Yuan, Ali H. Sayed
Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with n…
Linear Convergence of Primal-Dual Gradient Methods and their Performance in Distributed Optimization
Sulaiman A. Alghunaim, Ali H. Sayed
In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problem…
A Proximal Diffusion Strategy for Multi-Agent Optimization with Sparse Affine Constraints
Sulaiman A. Alghunaim, Kun Yuan, Ali H. Sayed
This work develops a proximal primal-dual decentralized strategy for multi-agent optimization problems that involve multiple coupled affine constraints, where each constraint may i…
Distributed Coupled Multi-Agent Stochastic Optimization
Sulaiman A. Alghunaim, Ali H. Sayed
This work develops effective distributed strategies for the solution of constrained multi-agent stochastic optimization problems with coupled parameters across the agents. In this…