1.8k citations · 7.5k across the 146 of their papers we have counts for
9 papers · 1 filter
Convergence to good non-optimal critical points in the training of neural networks: Gradient descent optimization with one random initialization overcomes all bad non-global local minima with high probability
Shokhrukh Ibragimov, Arnulf Jentzen, Adrian Riekert
Gradient descent (GD) methods for the training of artificial neural networks (ANNs) belong nowadays to the most heavily employed computational schemes in the digital world. Despite…
Blow up phenomena for gradient descent optimization methods in the training of artificial neural networks
Davide Gallon, Arnulf Jentzen, Felix Lindner
In this article we investigate blow up phenomena for gradient descent optimization methods in the training of artificial neural networks (ANNs). Our theoretical analysis is focused…
An efficient Monte Carlo scheme for Zakai equations
Christian Beck, Sebastian Becker, Patrick Cheridito +2
In this paper we develop a numerical method for efficiently approximating solutions of certain Zakai equations in high dimensions. The key idea is to transform a given Zakai SPDE i…
Gradient descent provably escapes saddle points in the training of shallow ReLU networks
Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek
Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms bypass so-called strict saddle points of the loss function. However, in…
Normalized gradient flow optimization in the training of ReLU artificial neural networks
Simon Eberle, Arnulf Jentzen, Adrian Riekert +1
The training of artificial neural networks (ANNs) is nowadays a highly relevant algorithmic procedure with many applications in science and industry. Roughly speaking, ANNs can be…
On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector
Arnulf Jentzen, Timo Kröger
It is an elementary fact in the scientific literature that the Lipschitz norm of the realization function of a feedforward fully-connected rectified linear unit (ReLU) artificial n…