First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms

Eric Lu

Stochastic Gradient Descent (SGD) methods see many uses in optimization problems. Modifications to the algorithm, such as momentum-based SGD methods have been known to produce bett…