17 citations · 32 across the 14 of their papers we have counts for
10 papers · 1 filter
Learning with Shallow Neural Networks on Cluster-Structured Features
Elisabetta Cornacchia, Laurent Massoulié
The success of deep learning in high-dimensional settings is often attributed to the presence of low-dimensional structure in real-world data. While standard theoretical models typ…
The Benefits of Temporal Correlations: SGD Learns k-Juntas from Random Walks Efficiently
Elisabetta Cornacchia, Dan Mikulincer, Elchanan Mossel
We study how temporal correlations in the data can make certain sparse learning problems efficiently learnable by gradient-based methods. Our focus is on Boolean k-juntas, a canoni…
Positive Distribution Shift as a Framework for Understanding Tractable Learning
Marko Medvedev, Idan Attias, Elisabetta Cornacchia +3
We study a setting where the goal is to learn a target function f(x) with respect to a target distribution D(x), but training is done on i.i.d. samples from a different training di…
Low-dimensional Functions are Efficiently Learnable under Randomly Biased Distributions
Elisabetta Cornacchia, Dan Mikulincer, Elchanan Mossel
The problem of learning single index and multi index models has gained significant interest as a fundamental task in high-dimensional statistics. Many recent works have analysed gr…
Learning High-Degree Parities: The Crucial Role of the Initialization
Emmanuel Abbe, Elisabetta Cornacchia, Jan Hązła +1
Parities have become a standard benchmark for evaluating learning algorithms. Recent works show that regular neural networks trained by gradient descent can efficiently learn degre…
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs
Emmanuel Abbe, Elisabetta Cornacchia, Aryo Lotfi
Experimental results have shown that curriculum learning, i.e., presenting simpler examples before more complex ones, can improve the efficiency of learning. Some recent theoretica…
A Mathematical Model for Curriculum Learning for Parities
Elisabetta Cornacchia, Elchanan Mossel
Curriculum learning (CL) - training using samples that are generated and presented in a meaningful order - was introduced in the machine learning context around a decade ago. While…
Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Emmanuel Abbe, Samy Bengio, Elisabetta Cornacchia +4
This paper considers the Pointer Value Retrieval (PVR) benchmark introduced in [ZRKB21], where a 'reasoning' function acts on a string of digits to produce the label. More generall…
An initial alignment between neural network and target is needed for gradient descent to learn
Emmanuel Abbe, Elisabetta Cornacchia, Jan Hązła +1
This paper introduces the notion of ``Initial Alignment'' (INAL) between a neural network at initialization and a target function. It is proved that if a network and a Boolean targ…
Regularization by Misclassification in ReLU Neural Networks
Elisabetta Cornacchia, Jan Hązła, Ido Nachum +1
We study the implicit bias of ReLU neural networks trained by a variant of SGD where at each step, the label is changed with probability to a random label (label smoothing bein…