30 citations · 46 across the 10 of their papers we have counts for
10 papers
Local Coverage Governs Memorization in Diffusion Models
Claudia Merger, Sebastian Goldt
Memorization in diffusion models is often treated as a global property of the model or dataset. In practice, however, a single diffusion model can simultaneously generate both memo…
Memorisation, convergence and generalisation in generative models
Antoine Maillard, Sebastian Goldt
Generative neural networks learn how to produce highly realistic images from a large, but finite number of examples - or do they simply memorise their training set? To settle this…
A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
Fabiola Ricci, Claudia Merger, Sebastian Goldt
Neural networks trained with gradient-based methods exhibit a strong simplicity bias: they learn simpler statistical features of their data before moving to more complex features.…
Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
Alessio Giorlandino, Sebastian Goldt, Antoine Maillard
Large language models excel at factual recall, yet the fundamental limits of storing input--output associations with neural networks remain unclear. We study these limits in a mini…
A theory of learning data statistics in diffusion models, from easy to hard
Lorenzo Bardone, Claudia Merger, Sebastian Goldt
While diffusion models have emerged as a powerful class of generative models, their learning dynamics remain poorly understood. We address this issue first by empirically showing t…
On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks
William T. Redman, Zhangyang Wang, Alessandro Ingrosso +1
Since its use in the Lottery Ticket Hypothesis, iterative magnitude pruning (IMP) has become a popular method for extracting sparse subnetworks that can be trained to high performa…
Neural networks trained with SGD learn distributions of increasing complexity
Maria Refinetti, Alessandro Ingrosso, Sebastian Goldt
The ability of deep neural networks to generalise well even when they interpolate their training data has been explained using various "simplicity biases". These theories postulate…
Data-driven emergence of convolutional structure in neural networks
Alessandro Ingrosso, Sebastian Goldt
Exploiting data invariances is crucial for efficient learning in both artificial and biological neural circuits. Understanding how neural networks can discover appropriate represen…
The dynamics of representation learning in shallow, non-linear autoencoders
Maria Refinetti, Sebastian Goldt
Autoencoders are the simplest neural network for unsupervised learning, and thus an ideal framework for studying feature learning. While a detailed understanding of the dynamics of…
Align, then memorise: the dynamics of learning with feedback alignment
Maria Refinetti, Stéphane d'Ascoli, Ruben Ohana +1
Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to the ubiquitous backpropagation algorithm for training deep neural networks. De…