36 citations · 163 across the 13 of their papers we have counts for
3 papers · 1 filter
On the different regimes of Stochastic Gradient Descent
Antonio Sclocchi, Matthieu Wyart
Modern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size , and the step siz…
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning
Antonio Sclocchi, Mario Geiger, Matthieu Wyart
Understanding when the noise in stochastic gradient descent (SGD) affects generalization of deep neural networks remains a challenge, complicated by the fact that networks can oper…
Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional data
Umberto M. Tomasini, Antonio Sclocchi, Matthieu Wyart
Recently, several theories including the replica method made predictions for the generalization error of Kernel Ridge Regression. In some regimes, they predict that the method has…