50 citations · 223 across the 27 of their papers we have counts for
4 papers · 1 filter
On Learning Gaussian Multi-index Models with Gradient Flow
Alberto Bietti, Joan Bruna, Loucas Pillaud-Vivien
We study gradient flow on the multi-index regression problem for high-dimensional Gaussian data. Multi-index functions consist of a composition of an unknown low-rank linear projec…
Scaling Laws for Associative Memories
Vivien Cabannes, Elvis Dohmatob, Alberto Bietti
Learning arguably involves the discovery and memorization of abstract rules. The aim of this paper is to study associative memory mechanisms. Our model is based on high-dimensional…
Birth of a Transformer: A Memory Viewpoint
Alberto Bietti, Vivien Cabannes, Diane Bouchacourt +2
Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their i…
The SSL Interplay: Augmentations, Inductive Bias, and Generalization
Vivien Cabannes, Bobak T. Kiani, Randall Balestriero +2
Self-supervised learning (SSL) has emerged as a powerful framework to learn representations from raw data without supervision. Yet in practice, engineers face issues such as instab…