48 citations · 50 across the 5 of their papers we have counts for
7 papers
First Learn, Then Memorize: The Spectral Bias of Diffusion Models
Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1
Diffusion models trained on a finite dataset first learn to generate novel, high-quality samples and only much later collapse onto their training set. We identify the mechanism beh…
Double Descent and Malign Overfitting in Diffusion Models
Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1
Conventional wisdom in deep learning holds that overparameterization---having more parameters than training samples ---is benign: larger models generalize better and, even w…
Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval
Antoine Maillard, Tony Bonnaire, Giulio Biroli
We consider the landscape of empirical risk minimization for high-dimensional Gaussian single-index models (generalized linear models). The objective is to recover an unknown signa…
Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training
Tony Bonnaire, Raphaël Urfin, Giulio Biroli +1
Diffusion models have achieved remarkable success across a wide range of generative tasks. A key challenge is understanding the mechanisms that prevent their memorization of traini…
The Role of the Time-Dependent Hessian in High-Dimensional Optimization
Tony Bonnaire, Giulio Biroli, Chiara Cammarota
Gradient descent is commonly used to find minima in rough landscapes, particularly in recent machine learning applications. However, a theoretical understanding of why good solutio…
Dynamical Regimes of Diffusion Models
Giulio Biroli, Tony Bonnaire, Valentin de Bortoli +1
Using statistical physics methods, we study generative diffusion models in the regime where the dimension of space and the number of data are large, and the score function has been…
High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics
Tony Bonnaire, Davide Ghio, Kamesh Krishnamurthy +3
In these lecture notes we present different methods and concepts developed in statistical physics to analyze gradient descent dynamics in high-dimensional non-convex landscapes. Ou…