Strong error analysis for the stochastic momentum optimizer

Davide Gallon, Arnulf Jentzen

Stochastic gradient descent (SGD) optimization schemes are the methods of choice for the optimization of deep neural networks (DNNs) in artificial intelligence (AI) systems. Often…