High-Probability Guarantees for SGD under $\beta$-Heavy-Tailed Gradient Noise

Qijun Tong, Masahiro Ikeda, Ryota Kawasumi

Stochastic gradient descent (SGD) is widely used to train machine learning models, but subsampling the training data introduces noise into its updates. The strength and applicabili…