44 citations · 199 across the 14 of their papers we have counts for
14 papers · 1 filter
Generalization Bounds for Stochastic Gradient Descent via Localized -Covers
Sejun Park, Umut Şimşekli, Murat A. Erdogdu
In this paper, we propose a new covering technique localized for the trajectories of SGD. This localization provides an algorithm-specific complexity measured by the covering numbe…
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks
Melih Barsbey, Milad Sefidgaran, Murat A. Erdogdu +2
Neural network compression techniques have become increasingly popular as they can drastically reduce the storage and computation requirements for very large networks. Recent empir…
Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections
Alexander Camuto, Xiaoyu Wang, Lingjiong Zhu +3
Gaussian noise injections (GNIs) are a family of simple and widely-used regularisation methods for training neural networks, where one injects additive or multiplicative Gaussian n…
Quantitative Propagation of Chaos for SGD in Wide Neural Networks
Valentin De Bortoli, Alain Durmus, Xavier Fontaine +1
In this paper, we investigate the limiting behavior of a continuous-time counterpart of the Stochastic Gradient Descent (SGD) algorithm applied to two-layer overparameterized neura…
Explicit Regularisation in Gaussian Noise Injections
Alexander Camuto, Matthew Willetts, Umut Şimşekli +2
We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have be…
Generalized Sliced Distances for Probability Distributions
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli +1
Probability metrics have become an indispensable part of modern statistics and machine learning, and they play a quintessential role in various applications, including statistical…