52 citations · 160 across the 10 of their papers we have counts for
10 papers
Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization
Ravi Srinivasan, Francesca Mignacco, Martino Sorbaro +4
"Forward-only" algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of ba…
Neural networks trained with SGD learn distributions of increasing complexity
Maria Refinetti, Alessandro Ingrosso, Sebastian Goldt
The ability of deep neural networks to generalise well even when they interpolate their training data has been explained using various "simplicity biases". These theories postulate…
Optimal learning rate schedules in high-dimensional non-convex optimization problems
Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli
Learning rate schedules are ubiquitously used to speed up and improve optimisation. Many different policies have been introduced on an empirical basis, and theoretical analyses hav…
Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension
Bruno Loureiro, Cédric Gerbelot, Maria Refinetti +2
From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered b…
The dynamics of representation learning in shallow, non-linear autoencoders
Maria Refinetti, Sebastian Goldt
Autoencoders are the simplest neural network for unsupervised learning, and thus an ideal framework for studying feature learning. While a detailed understanding of the dynamics of…
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed
Maria Refinetti, Sebastian Goldt, Florent Krzakala +1
A recent series of theoretical works showed that the dynamics of neural networks with a certain initialisation are well-captured by kernel methods. Concurrent empirical work demons…
Bootstrapping traceless symmetric scalars
Marten Reehorst, Maria Refinetti, Alessandro Vichi
We use numerical bootstrap techniques to study correlation functions of traceless symmetric tensors of with two indexes . We obtain upper bounds on operator dimensio…
Align, then memorise: the dynamics of learning with feedback alignment
Maria Refinetti, Stéphane d'Ascoli, Ruben Ohana +1
Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to the ubiquitous backpropagation algorithm for training deep neural networks. De…
Epidemic mitigation by statistical inference from contact tracing data
Antoine Baker, Indaco Biazzo, Alfredo Braunstein +10
Contact-tracing is an essential tool in order to mitigate the impact of pandemic such as the COVID-19. In order to achieve efficient and scalable contact-tracing in real time, digi…
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime
Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli +1
Deep neural networks can achieve remarkable generalization performances while interpolating the training data perfectly. Rather than the U-curve emblematic of the bias-variance tra…