48 citations · 56 across the 13 of their papers we have counts for
17 papers
First Learn, Then Memorize: The Spectral Bias of Diffusion Models
Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1
Diffusion models trained on a finite dataset first learn to generate novel, high-quality samples and only much later collapse onto their training set. We identify the mechanism beh…
Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data
Jérémie Klinger, Raphaël Urfin, Giulio Biroli +1
Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In practice this drift is modeled…
Double Descent and Malign Overfitting in Diffusion Models
Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1
Conventional wisdom in deep learning holds that overparameterization---having more parameters than training samples ---is benign: larger models generalize better and, even w…
WaiT for the Signal: Simple Frequency-Aware Flow-Matching
Krunoslav Lehman Pavasovic, Théophane Vallaeys, Stéphane Mallat +4
As image generation models scale to ever higher resolutions, global coherence, local detail, and texture fidelity become critical axes for generation quality. However, standard flo…
The critical slowing down in training diffusion models
Luca Maria Del Bono, Giulio Biroli, Patrick Charbonneau +1
Computational sampling has been central to the sciences since the mid-20th century. While machine-learning-based approaches have recently enabled major advances, their behavior rem…
Discontinuous BBP transitions
Dario Bocchi, Giulio Biroli, Chiara Cammarota +1
The Baik-Ben Arous-Peche (BBP) transition sets fundamental limits for detecting low-rank structure in noisy high-dimensional data and underlies a wide range of spectral methods in…
Theory of Speciation Transitions in Diffusion Models with General Class Structure
Beatrice Achilli, Marco Benedetti, Giulio Biroli +1
Diffusion Models generate data by reversing a stochastic diffusion process, progressively transforming noise into structured samples drawn from a target distribution. Recent theore…
Numerical investigation of the equilibrium Kauzmann transition in a two-dimensional atomistic glass
Gerhard Jung, Misaki Ozawa, Giulio Biroli +1
Dense liquids gradually transform into non-equilibrium amorphous solids as they pass through the experimental glass transition. Experimentally, ergodicity is lost because measureme…
Eigenvalue spectral tails and localization properties of asymmetric networks
Pietro Valigi, Joseph W. Baron, Izaak Neri +2
In contrast to the neatly bounded spectra of densely populated large random matrices, sparse random matrices often exhibit unbounded eigenvalue tails on the real and imaginary axis…
Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training
Tony Bonnaire, Raphaël Urfin, Giulio Biroli +1
Diffusion models have achieved remarkable success across a wide range of generative tasks. A key challenge is understanding the mechanisms that prevent their memorization of traini…
A Differentiable Rank-Based Objective For Better Feature Learning
Krunoslav Lehman Pavasovic, David Lopez-Paz, Giulio Biroli +1
In this paper, we leverage existing statistical methods to better understand feature learning from data. We tackle this by modifying the model-free variable selection method, Featu…
Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms
Krunoslav Lehman Pavasovic, Jakob Verbeek, Giulio Biroli +1
Classifier-Free Guidance (CFG) is a widely adopted technique in diffusion and flow-based generative models, enabling high-quality conditional generation. A key theoretical challeng…
Optimizing Noise Schedules of Generative Models in High Dimensionss
Santiago Aranguri, Giulio Biroli, Marc Mezard +1
Recent works have shown that diffusion models can undergo phase transitions, the resolution of which is needed for accurately generating samples. This has motivated the use of diff…
Kernel Density Estimators in Large Dimensions
Giulio Biroli, Marc Mézard
This paper studies Kernel Density Estimation for a high-dimensional distribution . Traditional approaches have focused on the limit of large number of data points and fix…
Cascade of phase transitions in the training of Energy-based models
Dimitrios Bachtis, Giulio Biroli, Aurélien Decelle +1
In this paper, we investigate the feature encoding process in a prototypical energy-based generative model, the Restricted Boltzmann Machine (RBM). We start with an analytical inve…
The Role of the Time-Dependent Hessian in High-Dimensional Optimization
Tony Bonnaire, Giulio Biroli, Chiara Cammarota
Gradient descent is commonly used to find minima in rough landscapes, particularly in recent machine learning applications. However, a theoretical understanding of why good solutio…
Dynamical Regimes of Diffusion Models
Giulio Biroli, Tony Bonnaire, Valentin de Bortoli +1
Using statistical physics methods, we study generative diffusion models in the regime where the dimension of space and the number of data are large, and the score function has been…