26 citations · 96 across the 22 of their papers we have counts for
5 papers · 1 filter
Asymptotic stability properties and a priori bounds for Adam and other gradient descent optimization methods
Steffen Dereich, Robin Graeber, Arnulf Jentzen +1
Gradient descent (GD) based optimization methods are these days the standard tools to train deep neural networks in artificial intelligence systems. In optimization procedures in d…
PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning
Arnulf Jentzen, Julian Kranz, Adrian Riekert
Averaging techniques such as Ruppert--Polyak averaging and exponential movering averaging (EMA) are powerful approaches to accelerate optimization procedures of stochastic gradient…
Sharp higher order convergence rates for the Adam optimizer
Steffen Dereich, Arnulf Jentzen, Adrian Riekert
Gradient descent based optimization methods are the methods of choice to train deep neural networks in machine learning. Beyond the standard gradient descent method, also suitable…
Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks
Thang Do, Arnulf Jentzen, Adrian Riekert
Despite the omnipresent use of stochastic gradient descent (SGD) optimization methods in the training of deep neural networks (DNNs), it remains, in basically all practically relev…
Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problems
Steffen Dereich, Arnulf Jentzen, Adrian Riekert
Deep learning methods - usually consisting of a class of deep neural networks (DNNs) trained by a stochastic gradient descent (SGD) optimization method - are nowadays omnipresent i…