52 citations · 160 across the 10 of their papers we have counts for
4 papers · 1 filter
Neural networks trained with SGD learn distributions of increasing complexity
Maria Refinetti, Alessandro Ingrosso, Sebastian Goldt
The ability of deep neural networks to generalise well even when they interpolate their training data has been explained using various "simplicity biases". These theories postulate…
Optimal learning rate schedules in high-dimensional non-convex optimization problems
Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli
Learning rate schedules are ubiquitously used to speed up and improve optimisation. Many different policies have been introduced on an empirical basis, and theoretical analyses hav…
Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension
Bruno Loureiro, Cédric Gerbelot, Maria Refinetti +2
From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered b…
The dynamics of representation learning in shallow, non-linear autoencoders
Maria Refinetti, Sebastian Goldt
Autoencoders are the simplest neural network for unsupervised learning, and thus an ideal framework for studying feature learning. While a detailed understanding of the dynamics of…