Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions

Martin Hutzenthaler, Arnulf Jentzen, Katharina Pohl +2

In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day…