9 citations · 32 across the 14 of their papers we have counts for
6 papers · 1 filter
Optimal Asymptotic Rates for (Stochastic) Gradient Descent under the Local PL-Condition: A Geometric Approach
Sebastian Kassing, Thomas Kruse
Stochastic gradient descent (SGD) has been studied extensively over the past decades due to its simplicity and broad applicability in machine learning. In this work, we analyze the…
ODE approximation for the Adam algorithm: General and overparametrized setting
Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
The Adam optimizer is currently presumably the most popular optimization method in deep learning. In this article we develop an ODE based method to study the Adam optimizer in a fa…
Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization
Sebastian Kassing, Simon Weissmann, Leif Döring
The present article studies the minimization of convex, L-smooth functions defined on a separable real Hilbert space. We analyze regularized stochastic gradient descent (reg-SGD),…
Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality
Sebastian Kassing, Simon Weissmann
In this work, we analyze the convergence of Polyak's heavy ball method in both continuous and discrete time for non-convex -objective functions satisfying the Polyak-Lojasiewi…
On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
In this article, we show existence of minimizers in the loss landscape for residual artificial neural networks (ANNs) with multi-dimensional input layer and one hidden layer with R…
Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting
Benjamin Gess, Sebastian Kassing
We prove explicit bounds on the exponential rate of convergence for the momentum stochastic gradient descent scheme (MSGD) for arbitrary, fixed hyperparameters (learning rate, fric…