5 papers
Sharp Spectral Thresholds for Multi-View Spiked Wigner Models
Xiaodong Yang, Subhabrata Sen, Yue M. Lu
Motivated by multimodal estimation, we study a multi-view spiked Wigner model in which several noisy matrix observations contain correlated latent spikes. We derive a spectral esti…
A Random Matrix Theory of Masked Self-Supervised Regression
Arie Wortsman Zurich, Federica Gerace, Bruno Loureiro +1
In the era of transformer models, masked self-supervised learning (SSL) has become a foundational training paradigm. A defining feature of masked SSL is that training aggregates pr…
When does Gaussian equivalence fail and how to fix it: Non-universal behavior of random features with quadratic scaling
Garrett G. Wen, Hong Hu, Yue M. Lu +2
A major effort in modern high-dimensional statistics has been devoted to the analysis of linear predictors trained on nonlinear feature embeddings via empirical risk minimization (…
Dynamical mean-field analysis of adaptive Langevin diffusions: Replica-symmetric fixed point and empirical Bayes
Zhou Fan, Justin Ko, Bruno Loureiro +2
In many applications of statistical estimation via sampling, one may wish to sample from a high-dimensional target distribution that is adaptively evolving to the samples already s…
Dynamical mean-field analysis of adaptive Langevin diffusions: Propagation-of-chaos and convergence of the linear response
Zhou Fan, Justin Ko, Bruno Loureiro +2
Motivated by an application to empirical Bayes learning in high-dimensional regression, we study a class of Langevin diffusions in a system with random disorder, where the drift co…