42 citations · 123 across the 15 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cond-mat.dis-nn2021★ 42 cited
The effective noise of Stochastic Gradient Descent
Francesca Mignacco, Pierfrancesco Urbani
Stochastic Gradient Descent (SGD) is the workhorse algorithm of deep learning technology. At each step of the training phase, a mini batch of samples is drawn from the training dat…
cond-mat.dis-nn2021★ 38 cited
Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
Francesca Mignacco, Pierfrancesco Urbani, Lenka Zdeborová
In this paper we investigate how gradient-based algorithms such as gradient descent, (multi-pass) stochastic gradient descent, its persistent variant, and the Langevin algorithm na…