5 papers
A Random Matrix Theory of Masked Self-Supervised Regression
Arie Wortsman Zurich, Federica Gerace, Bruno Loureiro +1
In the era of transformer models, masked self-supervised learning (SSL) has become a foundational training paradigm. A defining feature of masked SSL is that training aggregates pr…
Optical kernel machine with programmable nonlinearity
SeungYun Han, Fei Xia, Sylvain Gigan +2
Optical kernel machines offer high throughput and low latency. A nonlinear optical kernel can handle complex nonlinear data, but power consumption is typically high with the conven…
Breaking the curse of dimensionality for linear rules: optimal predictors over the ellipsoid
Alexis Ayme, Bruno Loureiro
In this work, we address the following question: What minimal structural assumptions are needed to prevent the degradation of statistical learning bounds with increasing dimensiona…
Dynamical mean-field analysis of adaptive Langevin diffusions: Replica-symmetric fixed point and empirical Bayes
Zhou Fan, Justin Ko, Bruno Loureiro +2
In many applications of statistical estimation via sampling, one may wish to sample from a high-dimensional target distribution that is adaptively evolving to the samples already s…
Dynamical mean-field analysis of adaptive Langevin diffusions: Propagation-of-chaos and convergence of the linear response
Zhou Fan, Justin Ko, Bruno Loureiro +2
Motivated by an application to empirical Bayes learning in high-dimensional regression, we study a class of Langevin diffusions in a system with random disorder, where the drift co…