134 citations · 553 across the 48 of their papers we have counts for
3 papers · 1 filter
Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic +1
We study convergence in high-probability of SGD-type methods in non-convex optimization and the presence of heavy-tailed noise. To combat the heavy-tailed noise, a general black-bo…
Distributed gradient methods under heavy-tailed communication noise
Manojlo Vukovic, Dusan Jakovetic, Dragana Bajovic +1
We consider a standard distributed optimization problem in which networked nodes collaboratively minimize the sum of their locally known convex costs. For this setting, we address…
Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
Shuhua Yu, Dusan Jakovetic, Soummya Kar
Heavy-tailed noise in nonconvex stochastic optimization has garnered increasing research interest, as empirical studies, including those on training attention models, suggest it is…