126 citations · 555 across the 57 of their papers we have counts for
4 papers · 1 filter
On the non-universality of deep learning: quantifying the cost of symmetry
Emmanuel Abbe, Enric Boix-Adsera
We prove limitations on what neural networks trained by noisy gradient descent (GD) can efficiently learn. Our results apply whenever GD training is equivariant, which holds for ma…
Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Emmanuel Abbe, Samy Bengio, Elisabetta Cornacchia +4
This paper considers the Pointer Value Retrieval (PVR) benchmark introduced in [ZRKB21], where a 'reasoning' function acts on a string of digits to produce the label. More generall…
An initial alignment between neural network and target is needed for gradient descent to learn
Emmanuel Abbe, Elisabetta Cornacchia, Jan Hązła +1
This paper introduces the notion of ``Initial Alignment'' (INAL) between a neural network at initialization and a target function. It is proved that if a network and a Boolean targ…
The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks
Emmanuel Abbe, Enric Boix-Adsera, Theodor Misiakiewicz
It is currently known how to characterize functions that neural networks can learn with SGD for two extremal parameterizations: neural networks in the linear regime, and neural net…