20 citations · 48 across the 14 of their papers we have counts for
7 papers · 1 filter
Analysis of a Class of Stochastic Component-Wise Soft-Clipping Schemes
Måns Williamson, Monika Eisenmann, Tony Stillfjord
Choosing the optimization algorithm that performs best on a given machine learning problem is often delicate, and there is no guarantee that current state-of-the-art algorithms wil…
Almost sure convergence of stochastic Hamiltonian descent methods
Måns Williamson, Tony Stillfjord
Gradient normalization and soft clipping are two popular techniques for tackling instability issues and improving convergence of stochastic gradient descent (SGD) with momentum. In…
SRKCD: a stabilized Runge-Kutta method for stochastic optimization
Tony Stillfjord, Måns Williamson
We introduce a family of stochastic optimization methods based on the Runge-Kutta-Chebyshev (RKC) schemes. The RKC methods are explicit methods originally designed for solving stif…
Sub-linear convergence of a tamed stochastic gradient descent method in Hilbert space
Monika Eisenmann, Tony Stillfjord
In this paper, we introduce the tamed stochastic gradient descent method (TSGD) for optimization problems. Inspired by the tamed Euler scheme, which is a commonly used method withi…
Sub-linear convergence of a stochastic proximal iteration method in Hilbert space
Monika Eisenmann, Tony Stillfjord, Måns Williamson
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the…
Singular value decay of operator-valued differential Lyapunov and Riccati equations
Tony Stillfjord
We consider operator-valued differential Lyapunov and Riccati equations, where the operators and may be relatively unbounded with respect to (in the standard notation).…
Adaptive high-order splitting schemes for large-scale differential Riccati equations
Tony Stillfjord
We consider high-order splitting schemes for large-scale differential Riccati equations. Such equations arise in many different areas and are especially important within the field…