2 citations · 3 across the 8 of their papers we have counts for
3 papers · 1 filter
Learning High-Degree Parities: The Crucial Role of the Initialization
Emmanuel Abbe, Elisabetta Cornacchia, Jan Hązła +1
Parities have become a standard benchmark for evaluating learning algorithms. Recent works show that regular neural networks trained by gradient descent can efficiently learn degre…
An initial alignment between neural network and target is needed for gradient descent to learn
Emmanuel Abbe, Elisabetta Cornacchia, Jan Hązła +1
This paper introduces the notion of ``Initial Alignment'' (INAL) between a neural network at initialization and a target function. It is proved that if a network and a Boolean targ…
Regularization by Misclassification in ReLU Neural Networks
Elisabetta Cornacchia, Jan Hązła, Ido Nachum +1
We study the implicit bias of ReLU neural networks trained by a variant of SGD where at each step, the label is changed with probability to a random label (label smoothing bein…