30 citations · 46 across the 10 of their papers we have counts for
7 papers · 1 filter
Memorisation, convergence and generalisation in generative models
Antoine Maillard, Sebastian Goldt
Generative neural networks learn how to produce highly realistic images from a large, but finite number of examples - or do they simply memorise their training set? To settle this…
A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
Fabiola Ricci, Claudia Merger, Sebastian Goldt
Neural networks trained with gradient-based methods exhibit a strong simplicity bias: they learn simpler statistical features of their data before moving to more complex features.…
Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
Alessio Giorlandino, Sebastian Goldt, Antoine Maillard
Large language models excel at factual recall, yet the fundamental limits of storing input--output associations with neural networks remain unclear. We study these limits in a mini…
A theory of learning data statistics in diffusion models, from easy to hard
Lorenzo Bardone, Claudia Merger, Sebastian Goldt
While diffusion models have emerged as a powerful class of generative models, their learning dynamics remain poorly understood. We address this issue first by empirically showing t…
Neural networks trained with SGD learn distributions of increasing complexity
Maria Refinetti, Alessandro Ingrosso, Sebastian Goldt
The ability of deep neural networks to generalise well even when they interpolate their training data has been explained using various "simplicity biases". These theories postulate…
The dynamics of representation learning in shallow, non-linear autoencoders
Maria Refinetti, Sebastian Goldt
Autoencoders are the simplest neural network for unsupervised learning, and thus an ideal framework for studying feature learning. While a detailed understanding of the dynamics of…
Align, then memorise: the dynamics of learning with feedback alignment
Maria Refinetti, Stéphane d'Ascoli, Ruben Ohana +1
Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to the ubiquitous backpropagation algorithm for training deep neural networks. De…