1 citations · 1 across the 16 of their papers we have counts for
8 papers · 1 filter
High-dimensional Analysis of Synthetic Data Selection
Parham Rezaei, Filip Kovacevic, Francesco Locatello +1
Despite the progress in the development of generative models, their usefulness in creating synthetic data that improve prediction performance of classifiers has been put into quest…
Optimal Regularization for Performative Learning
Edwige Cyffers, Alireza Mirrokni, Marco Mondelli
In performative learning, the data distribution reacts to the deployed model - for example, because strategic users adapt their features to game it - which creates a more complex d…
Attention with Trained Embeddings Provably Selects Important Tokens
Diyuan Wu, Aleksandr Shevchenko, Samet Oymak +1
Token embeddings play a crucial role in language modeling but, despite this practical relevance, their theoretical understanding remains limited. Our paper addresses the gap by cha…
Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers
Peter Súkeník, Christoph H. Lampert, Marco Mondelli
The empirical emergence of neural collapse -- a surprising symmetry in the feature representations of the training data in the penultimate layer of deep neural networks -- has spur…
Test-Time Training Provably Improves Transformers as In-context Learners
Halil Alperen Gozeten, M. Emrullah Ildiz, Xuechen Zhang +3
Test-time training (TTT) methods explicitly update the weights of a model to adapt to the specific test instance, and they have found success in a variety of settings, including mo…
Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime
Diyuan Wu, Marco Mondelli
Neural Collapse is a phenomenon where the last-layer representations of a well-trained neural network converge to a highly structured geometry. In this paper, we focus on its first…
Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery
Filip Kovačević, Yihan Zhang, Marco Mondelli
Multi-index models provide a popular framework to investigate the learnability of functions with low-dimensional structure and, also due to their connections with neural networks,…
Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization
Simone Bombari, Marco Mondelli
Learning models have been shown to rely on spurious correlations between non-predictive features and the associated labels in the training data, with negative implications on robus…