5 papers
Regularization by Misclassification in ReLU Neural Networks
Elisabetta Cornacchia, Jan Hązła, Ido Nachum +1
We study the implicit bias of ReLU neural networks trained by a variant of SGD where at each step, the label is changed with probability to a random label (label smoothing bein…
On Symmetry and Initialization for Neural Networks
Ido Nachum, Amir Yehudayoff
This work provides an additional step in the theoretical understanding of neural networks. We consider neural networks with one hidden layer and show that when learning symmetric f…
Average-Case Information Complexity of Learning
Ido Nachum, Amir Yehudayoff
How many bits of information are revealed by a learning algorithm for a concept class of VC-dimension ? Previous works have shown that even for the amount of information m…
On the Perceptron's Compression
Shay Moran, Ido Nachum, Itai Panasoff +1
We study and provide exposition to several phenomena that are related to the perceptron's compression. One theme concerns modifications of the perceptron algorithm that yield bette…
A Direct Sum Result for the Information Complexity of Learning
Ido Nachum, Jonathan Shafer, Amir Yehudayoff
How many bits of information are required to PAC learn a class of hypotheses of VC dimension ? The mathematical setting we follow is that of Bassily et al. (2018), where the val…