9 citations · 32 across the 14 of their papers we have counts for
16 papers
Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
Sebastian Kassing
We construct a continuous function on for which the infimum of the -approximation error over the class of ReLU networks with two hidden layers and architecture $(2,…
Optimal Asymptotic Rates for (Stochastic) Gradient Descent under the Local PL-Condition: A Geometric Approach
Sebastian Kassing, Thomas Kruse
Stochastic gradient descent (SGD) has been studied extensively over the past decades due to its simplicity and broad applicability in machine learning. In this work, we analyze the…
The Role of Target Update Frequencies in Q-Learning
Simon Weissmann, Tilman Aach, Benedikt Wille +2
The target network update frequency (TUF) is a central stabilization mechanism in (deep) Q-learning. However, their selection remains poorly understood and is often treated merely…
An Approximate Ascent Approach To Prove Convergence of PPO
Leif Doering, Daniel Schmidt, Moritz Melcher +4
Proximal Policy Optimization (PPO) is among the most widely used deep reinforcement learning algorithms, yet its theoretical foundations remain incomplete. Most importantly, conver…
ODE approximation for the Adam algorithm: General and overparametrized setting
Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
The Adam optimizer is currently presumably the most popular optimization method in deep learning. In this article we develop an ODE based method to study the Adam optimizer in a fa…
Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization
Sebastian Kassing, Simon Weissmann, Leif Döring
The present article studies the minimization of convex, L-smooth functions defined on a separable real Hilbert space. We analyze regularized stochastic gradient descent (reg-SGD),…
Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality
Sebastian Kassing, Simon Weissmann
In this work, we analyze the convergence of Polyak's heavy ball method in both continuous and discrete time for non-convex -objective functions satisfying the Polyak-Lojasiewi…
Stochastic Modified Flows for Riemannian Stochastic Gradient Descent
Benjamin Gess, Sebastian Kassing, Nimit Rana
We give quantitative estimates for the rate of convergence of Riemannian stochastic gradient descent (RSGD) to Riemannian gradient flow and to a diffusion process, the so-called Ri…
On the existence of optimal shallow feedforward networks with ReLU activation
Steffen Dereich, Sebastian Kassing
We prove existence of global minima in the loss landscape for the approximation of continuous target functions using shallow feedforward artificial neural networks with ReLU activa…
Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent
Benjamin Gess, Sebastian Kassing, Vitalii Konarovskyi
We propose new limiting dynamics for stochastic gradient descent in the small learning rate regime called stochastic modified flows. These SDEs are driven by a cylindrical Brownian…
On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
In this article, we show existence of minimizers in the loss landscape for residual artificial neural networks (ANNs) with multi-dimensional input layer and one hidden layer with R…
Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting
Benjamin Gess, Sebastian Kassing
We prove explicit bounds on the exponential rate of convergence for the momentum stochastic gradient descent scheme (MSGD) for arbitrary, fixed hyperparameters (learning rate, fric…
On minimal representations of shallow ReLU networks
S. Dereich, S. Kassing
The realization function of a shallow ReLU network is a continuous and piecewise affine function , where the domain is partitioned by a…
Cooling down stochastic differential equations: almost sure convergence
S. Dereich, S. Kassing
We consider almost sure convergence of the SDE under the existence of a -Lyapunov function . More explicitly, we show t…
Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes
Steffen Dereich, Sebastian Kassing
In this article, we consider convergence of stochastic gradient descent schemes (SGD), including momentum stochastic gradient descent (MSGD), under weak assumptions on the underlyi…
Central limit theorems for stochastic gradient descent with averaging for stable manifolds
Steffen Dereich, Sebastian Kassing
In this article we establish new central limit theorems for Ruppert-Polyak averaged stochastic gradient descent schemes. Compared to previous work we do not assume that convergence…