44 citations · 543 across the 47 of their papers we have counts for
8 papers · 1 filter
Quantitative Propagation of Chaos for SGD in Wide Neural Networks
Valentin De Bortoli, Alain Durmus, Xavier Fontaine +1
In this paper, we investigate the limiting behavior of a continuous-time counterpart of the Stochastic Gradient Descent (SGD) algorithm applied to two-layer overparameterized neura…
Explicit Regularisation in Gaussian Noise Injections
Alexander Camuto, Matthew Willetts, Umut Şimşekli +2
We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have be…
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks
Umut Şimşekli, Ozan Sener, George Deligiannidis +1
Despite its success in a wide range of applications, characterizing the generalization properties of stochastic gradient descent (SGD) in non-convex deep learning problems is still…
The Heavy-Tail Phenomenon in SGD
Mert Gurbuzbalaban, Umut Şimşekli, Lingjiong Zhu
In recent years, various notions of capacity and complexity have been proposed for characterizing the generalization properties of stochastic gradient descent (SGD) in deep learnin…
Synchronizing Probability Measures on Rotations via Optimal Transport
Tolga Birdal, Michael Arbel, Umut Şimşekli +1
We introduce a new paradigm, , for synchronizing graphs with measure-valued edges. We formulate this problem as maximization of the cycle-consiste…
Statistical and Topological Properties of Sliced Probability Divergences
Kimia Nadjahi, Alain Durmus, Lénaïc Chizat +3
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and…
Generalized Sliced Distances for Probability Distributions
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli +1
Probability metrics have become an indispensable part of modern statistics and machine learning, and they play a quintessential role in various applications, including statistical…
Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise
Umut Şimşekli, Lingjiong Zhu, Yee Whye Teh +1
Stochastic gradient descent with momentum (SGDm) is one of the most popular optimization algorithms in deep learning. While there is a rich theory of SGDm for convex problems, the…