Spectral alignment of stochastic gradient descent for high-dimensional classification tasks

Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang +1

We rigorously study the relation between the training dynamics via stochastic gradient descent (SGD) and the spectra of empirical Hessian and gradient matrices. We prove that in tw…