36 citations · 163 across the 13 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 28 cited
On the different regimes of Stochastic Gradient Descent
Antonio Sclocchi, Matthieu Wyart
Modern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size , and the step siz…
cs.LG2023★ 2 cited
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning
Antonio Sclocchi, Mario Geiger, Matthieu Wyart
Understanding when the noise in stochastic gradient descent (SGD) affects generalization of deep neural networks remains a challenge, complicated by the fact that networks can oper…