419 citations · 3.1k across the 71 of their papers we have counts for
7 papers · 1 filter
Learning Boolean Circuits with Neural Networks
Eran Malach, Shai Shalev-Shwartz
While on some natural distributions, neural-networks are trained efficiently using gradient-based algorithms, it is known that learning them is computationally hard in the worst-ca…
The Implicit Bias of Depth: How Incremental Learning Drives Generalization
Daniel Gissin, Shai Shalev-Shwartz, Amit Daniely
A leading hypothesis for the surprising generalization of neural networks is that the dynamics of gradient descent bias the model towards simple solutions, by searching through the…
SenseBERT: Driving Some Sense into BERT
Yoav Levine, Barak Lenz, Or Dagan +6
The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding. However, existing self-supervision t…
Discriminative Active Learning
Daniel Gissin, Shai Shalev-Shwartz
We propose a new batch mode active learning algorithm designed for neural networks and large query batch sizes. The method, Discriminative Active Learning (DAL), poses active learn…
Decoupling Gating from Linearity
Jonathan Fiat, Eran Malach, Shai Shalev-Shwartz
ReLU neural-networks have been in the focus of many recent theoretical works, trying to explain their empirical success. Nonetheless, there is still a gap between current theoretic…
Is Deeper Better only when Shallow is Good?
Eran Malach, Shai Shalev-Shwartz
Understanding the power of depth in feed-forward neural networks is an ongoing challenge in the field of deep learning theory. While current works account for the importance of dep…