4 papers
The Advective Fisher-Rao Geometry of Deterministic Measure Transport
Benjamin Gess, Johannes Müller
A novel advective Fisher-Rao metric is introduced for optimization tasks on paths of probability measures governed by the continuity equation. This metric is shown to lead to optim…
Large Spikes in Stochastic Gradient Descent: A Large-Deviations View
Benjamin Gess, Daniel Heydecker
Large loss spikes in stochastic gradient descent are studied through a rigorous large-deviations analysis for a shallow, fully connected network in the NTK scaling. In contrast to…
THINNs: Thermodynamically Informed Neural Networks
Javier Castro, Benjamin Gess
Physics-Informed Neural Networks (PINNs) are a class of deep learning models aiming to approximate solutions of PDEs by training neural networks to minimize the residual of the equ…
Stochastic Modified Flows for Riemannian Stochastic Gradient Descent
Benjamin Gess, Sebastian Kassing, Nimit Rana
We give quantitative estimates for the rate of convergence of Riemannian stochastic gradient descent (RSGD) to Riemannian gradient flow and to a diffusion process, the so-called Ri…