Revisiting SGD with Increasingly Weighted Averaging: Optimization and Generalization Perspectives

Zhishuai Guo, Yan Yan, Tianbao Yang

Stochastic gradient descent (SGD) has been widely studied in the literature from different angles, and is commonly employed for solving many big data machine learning problems. How…