10 papers
Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
Aleksandar Armacki, Dragana BajoviÄ, DuÅ¡an JakovetiÄ +2
The study of tail behaviour of SGD-induced processes has been attracting a lot of interest, due to offering strong guarantees with respect to individual runs of an algorithm. While…
Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
Shuhua Yu, Dusan Jakovetic, Soummya Kar
Heavy-tailed noise in nonconvex stochastic optimization has garnered increasing research interest, as empirical studies, including those on training attention models, suggest it is…
Tackling heavy-tailed noise in distributed estimation: Asymptotic performance and tradeoffs
Dragana Bajovic, Dusan Jakovetic, Soummya Kar +1
We present an algorithm for distributed estimation of an unknown vector parameter in the presence of heavy-tailed observation and communicati…
Distributed Gradient Clustering: Convergence and the Effect of Initialization
Aleksandar Armacki, Himkant Sharma, Dragana BajoviÄ +3
We study the effects of center initialization on the performance of a family of distributed gradient-based clustering algorithms introduced in [1], that work over connected network…
Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic +1
We study convergence in high-probability of SGD-type methods in non-convex optimization and the presence of heavy-tailed noise. To combat the heavy-tailed noise, a general black-bo…
Distributed gradient methods under heavy-tailed communication noise
Manojlo Vukovic, Dusan Jakovetic, Dragana Bajovic +1
We consider a standard distributed optimization problem in which networked nodes collaboratively minimize the sum of their locally known convex costs. For this setting, we address…