A Random Matrix Theory of Masked Self-Supervised Regression

Arie Wortsman Zurich, Federica Gerace, Bruno Loureiro +1

In the era of transformer models, masked self-supervised learning (SSL) has become a foundational training paradigm. A defining feature of masked SSL is that training aggregates pr…