6 papers
Local Coverage Governs Memorization in Diffusion Models
Claudia Merger, Sebastian Goldt
Memorization in diffusion models is often treated as a global property of the model or dataset. In practice, however, a single diffusion model can simultaneously generate both memo…
A theory of learning data statistics in diffusion models, from easy to hard
Lorenzo Bardone, Claudia Merger, Sebastian Goldt
While diffusion models have emerged as a powerful class of generative models, their learning dynamics remain poorly understood. We address this issue first by empirically showing t…
Memorisation, convergence and generalisation in generative models
Antoine Maillard, Sebastian Goldt
Generative neural networks learn how to produce highly realistic images from a large, but finite number of examples - or do they simply memorise their training set? To settle this…
A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
Fabiola Ricci, Claudia Merger, Sebastian Goldt
Neural networks trained with gradient-based methods exhibit a strong simplicity bias: they learn simpler statistical features of their data before moving to more complex features.…
Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
Alessio Giorlandino, Sebastian Goldt, Antoine Maillard
Large language models demonstrate remarkable ability in factual recall, yet the fundamental limits of storing and retrieving input--output associations with neural networks remain…
On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks
William T. Redman, Zhangyang Wang, Alessandro Ingrosso +1
Since its use in the Lottery Ticket Hypothesis, iterative magnitude pruning (IMP) has become a popular method for extracting sparse subnetworks that can be trained to high performa…