9 papers
High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
Aukosh Jagannath, Taj Jones-McCormick, Varnan Sarangian
We develop a high-dimensional scaling limit for Stochastic Gradient Descent with Polyak Momentum (SGD-M) and adaptive step-sizes. This provides a framework to rigourously compare o…
Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions
Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang +1
We study the local geometry of empirical risks in high dimensions via the spectral theory of their Hessian and information matrices. We focus on settings where the data, $(Y_\ell)_…
Universality of high-dimensional scaling limits of stochastic gradient descent
Reza Gheissari, Aukosh Jagannath
We consider statistical tasks in high dimensions whose loss depends on the data only through its projection into a fixed-dimensional subspace spanned by the parameter vectors and c…
Pseudo-Maximum Likelihood Theory for High-Dimensional Rank One Inference
Curtis Grant, Aukosh Jagannath, Justin Ko
We develop a pseudo-likelihood theory for rank one matrix estimation problems in the high dimensional limit. We prove a variational principle for the limiting pseudo-maximum likeli…
Differentially private multivariate medians
Kelly Ramsay, Aukosh Jagannath, Shoja'eddin Chenouri
Statistical tools which satisfy rigorous privacy guarantees are necessary for modern data analysis. It is well-known that robustness against contamination is linked to differential…
Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Models
Taj Jones-McCormick, Aukosh Jagannath, Subhabrata Sen
Unsupervised pre-training and transfer learning are commonly used techniques to initialize training algorithms for neural networks, particularly in settings with limited labeled da…