2 citations · 3 across the 14 of their papers we have counts for
3 papers · 1 filter
Convergence rates for the Adam optimizer
Steffen Dereich, Arnulf Jentzen
Stochastic gradient descent (SGD) optimization methods are nowadays the method of choice for the training of deep neural networks (DNNs) in artificial intelligence systems. In prac…
Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates
Steffen Dereich, Robin Graeber, Arnulf Jentzen
Deep learning algorithms - typically consisting of a class of deep neural networks trained by a stochastic gradient descent (SGD) optimization method - are nowadays the key ingredi…
Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
Steffen Dereich, Arnulf Jentzen, Adrian Riekert
The standard stochastic gradient descent (SGD) optimization method, as well as adaptive methods such as the Adam optimizer fail to converge if the learning rates do not converge to…