2 citations · 2 across the 8 of their papers we have counts for
6 papers · 1 filter
High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking
Aleksandar Armacki, Haoyuan Cai, Ali H. Sayed
We study high-probability (HP) convergence guarantees in decentralized stochastic optimization, where multiple agents collaborate to jointly train a model over a network. Existing…
Distributed Gradient Clustering: Convergence and the Effect of Initialization
Aleksandar Armacki, Himkant Sharma, Dragana Bajović +3
We study the effects of center initialization on the performance of a family of distributed gradient-based clustering algorithms introduced in [1], that work over connected network…
Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
Aleksandar Armacki, Dragana Bajović, Dušan Jakovetić +2
The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm. While…
High-Probability Convergence Guarantees of Decentralized SGD
Aleksandar Armacki, Ali H. Sayed
Convergence in high-probability (HP) has attracted increasing interest, due to implying exponentially decaying tail bounds and strong guarantees for individual runs of an algorithm…
Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry
Aleksandar Armacki, Shuhua Yu, Dragana Bajovic +2
We study large deviation upper bounds and mean-squared error (MSE) guarantees of a general framework of nonlinear stochastic gradient methods in the online setting, in the presence…
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
Aleksandar Armacki, Shuhua Yu, Pranay Sharma +4
We study high-probability convergence in online learning, in the presence of heavy-tailed noise. To combat the heavy tails, a general framework of nonlinear SGD methods is consider…