Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization

Yadong Mu, Wei Liu, Wei Fan

Stochastic gradient descent (SGD) holds as a classical method to build large scale machine learning models over big data. A stochastic gradient is typically calculated from a limit…