1 citations · 1 across the 4 of their papers we have counts for
4 papers
Analysis of a Class of Stochastic Component-Wise Soft-Clipping Schemes
Måns Williamson, Monika Eisenmann, Tony Stillfjord
Choosing the optimization algorithm that performs best on a given machine learning problem is often delicate, and there is no guarantee that current state-of-the-art algorithms wil…
Almost sure convergence of stochastic Hamiltonian descent methods
Måns Williamson, Tony Stillfjord
Gradient normalization and soft clipping are two popular techniques for tackling instability issues and improving convergence of stochastic gradient descent (SGD) with momentum. In…
SRKCD: a stabilized Runge-Kutta method for stochastic optimization
Tony Stillfjord, Måns Williamson
We introduce a family of stochastic optimization methods based on the Runge-Kutta-Chebyshev (RKC) schemes. The RKC methods are explicit methods originally designed for solving stif…
Sub-linear convergence of a stochastic proximal iteration method in Hilbert space
Monika Eisenmann, Tony Stillfjord, Måns Williamson
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the…