4 papers
Universality of high-dimensional scaling limits of stochastic gradient descent
Reza Gheissari, Aukosh Jagannath
We consider statistical tasks in high dimensions whose loss depends on the data only through its projection into a fixed-dimensional subspace spanned by the parameter vectors and c…
Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions
Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang +1
We study the local geometry of empirical risks in high dimensions via the spectral theory of their Hessian and information matrices. We focus on settings where the data, $(Y_\ell)_…
Finding planted cliques using gradient descent
Reza Gheissari, Aukosh Jagannath, Yiming Xu
The planted clique problem is a paradigmatic model of statistical-to-computational gaps: the planted clique is information-theoretically detectable if its size but…
Spectral alignment of stochastic gradient descent for high-dimensional classification tasks
Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang +1
We rigorously study the relation between the training dynamics via stochastic gradient descent (SGD) and the spectra of empirical Hessian and gradient matrices. We prove that in tw…