NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer

Valentin Leplat, Daniil Merkulov, Aleksandr Katrutsa +3

Classical machine learning models such as deep neural networks are usually trained by using Stochastic Gradient Descent-based (SGD) algorithms. The classical SGD can be interpreted…