9 papers
Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities
Devon Jarvis, Richard Klein, Benjamin Rosman +2
Model collapse, the degradation in performance that arises when generative models are trained on the outputs of prior models, is an increasing concern as artificially generated con…
Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks
Jie Huang, Bruno Loureiro, Stefano Sarao Mannelli
We study the population loss landscape of two-layer ReLU networks of the form in a realisable teacher-student setting with Gaussian covaria…
The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models
Flavio Nicoletti, Chenxiao Ma, Enrico Ventura +2
Real-world datasets are inherently heterogeneous, yet how per-class structural differences and sampling imbalance shape the training dynamics of diffusion models-and potentially ex…
Reducing Recurrent Competitive Epidemics via Dynamic Resource Allocation
Argyris Kalogeratos, Gaspard Abel, Stefano Sarao Mannelli
Motivated by scenarios of epidemic competition, as well as how social contagions spread at the level of individuals, this work considers the competition between two conflicting nod…
Optimal Protocols for Continual Learning via Statistical Physics and Control Theory
Francesco Mori, Stefano Sarao Mannelli, Francesca Mignacco
Artificial neural networks often struggle with catastrophic forgetting when learning multiple tasks sequentially, as training on new tasks degrades the performance on previously le…
A Theory of Initialisation's Impact on Specialisation
Devon Jarvis, Sebastian Lee, Clémentine Carla Juliette Dominé +2
Prior work has demonstrated a consistent tendency in neural networks engaged in continual learning tasks, wherein intermediate task similarity results in the highest levels of cata…