52 citations · 56 across the 2 of their papers we have counts for
4 papers
Optimal learning rate schedules in high-dimensional non-convex optimization problems
Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli
Learning rate schedules are ubiquitously used to speed up and improve optimisation. Many different policies have been introduced on an empirical basis, and theoretical analyses hav…
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed
Maria Refinetti, Sebastian Goldt, Florent Krzakala +1
A recent series of theoretical works showed that the dynamics of neural networks with a certain initialisation are well-captured by kernel methods. Concurrent empirical work demons…
Align, then memorise: the dynamics of learning with feedback alignment
Maria Refinetti, Stéphane d'Ascoli, Ruben Ohana +1
Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to the ubiquitous backpropagation algorithm for training deep neural networks. De…
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime
Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli +1
Deep neural networks can achieve remarkable generalization performances while interpolating the training data perfectly. Rather than the U-curve emblematic of the bias-variance tra…