38 citations · 113 across the 6 of their papers we have counts for
3 papers · 1 filter
Gradient descent provably escapes saddle points in the training of shallow ReLU networks
Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek
Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms bypass so-called strict saddle points of the loss function. However, in…
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions
Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek
In this paper, we analyze the landscape of the true loss of neural networks with one hidden layer and ReLU, leaky ReLU, or quadratic activation. In all three cases, we provide a co…
Non-convergence of stochastic gradient descent in the training of deep neural networks
Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek
Deep neural networks have successfully been trained in various application areas with stochastic gradient descent. However, there exists no rigorous mathematical explanation why th…