9 citations · 32 across the 14 of their papers we have counts for
4 papers · 1 filter
On the existence of optimal shallow feedforward networks with ReLU activation
Steffen Dereich, Sebastian Kassing
We prove existence of global minima in the loss landscape for the approximation of continuous target functions using shallow feedforward artificial neural networks with ReLU activa…
Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent
Benjamin Gess, Sebastian Kassing, Vitalii Konarovskyi
We propose new limiting dynamics for stochastic gradient descent in the small learning rate regime called stochastic modified flows. These SDEs are driven by a cylindrical Brownian…
On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
In this article, we show existence of minimizers in the loss landscape for residual artificial neural networks (ANNs) with multi-dimensional input layer and one hidden layer with R…
Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting
Benjamin Gess, Sebastian Kassing
We prove explicit bounds on the exponential rate of convergence for the momentum stochastic gradient descent scheme (MSGD) for arbitrary, fixed hyperparameters (learning rate, fric…