Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance

Hongjian Wang, Mert Gürbüzbalaban, Lingjiong Zhu +2

Recent studies have provided both empirical and theoretical evidence illustrating that heavy tails can emerge in stochastic gradient descent (SGD) in various scenarios. Such heavy…