On the Trajectories of SGD Without Replacement

Pierfrancesco Beneventano

This article examines the implicit regularization effect of Stochastic Gradient Descent (SGD). We consider the case of SGD without replacement, the variant typically used to optimi…