126 citations · 244 across the 10 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2022★ 10 cited
Non-Convergence and Limit Cycles in the Adam optimizer
Sebastian Bock, Martin Georg Weiß
One of the most popular training algorithms for deep neural networks is the Adaptive Moment Estimation (Adam) introduced by Kingma and Ba. Despite its success in many applications…
cs.LG2021
Local Convergence of Adaptive Gradient Descent Optimizers
Sebastian Bock, Martin Georg Weiß
Adaptive Moment Estimation (ADAM) is a very popular training algorithm for deep neural networks and belongs to the family of adaptive gradient descent optimizers. However to the be…
cs.LG2018★ 126 cited
An improvement of the convergence proof of the ADAM-Optimizer
Sebastian Bock, Josef Goppold, Martin Weiß
A common way to train neural networks is the Backpropagation. This algorithm includes a gradient descent method, which needs an adaptive step size. In the area of neural networks,…