Weighted Averaged Stochastic Gradient Descent: Asymptotic Normality and Optimality

Ziyang Wei, Wanrong Zhu, Wei Biao Wu

Stochastic Gradient Descent (SGD) is one of the most popular algorithms in statistical and machine learning due to its computational and memory efficiency. Various averaging scheme…