Adaptive Heavy-Tailed Stochastic Gradient Descent

Bodu Gong, Gustavo Enrique Batista, Pierre Lafaye de Micheaux

In the era of large-scale neural network models, optimization algorithms often struggle with generalization due to an overreliance on training loss. One key insight widely accepted…