29 citations · 91 across the 43 of their papers we have counts for
8 papers · 1 filter
Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
Spencer Frei, Gal Vardi, Peter L. Bartlett +2
The implicit biases of gradient-based optimization algorithms are conjectured to be a major factor in the success of modern deep learning. In this work, we investigate the implicit…
On the Implicit Bias in Deep-Learning Algorithms
Gal Vardi
Gradient-based deep-learning algorithms exhibit remarkable performance in practice, but it is not well-understood why they are able to generalize despite having more parameters tha…
Reconstructing Training Data from Trained Neural Networks
Niv Haim, Gal Vardi, Gilad Yehudai +2
Understanding to what extent neural networks memorize training data is an intriguing question with practical and theoretical implications. In this paper we show that in some cases…
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias
Itay Safran, Gal Vardi, Jason D. Lee
We study the dynamics and implicit bias of gradient flow (GF) on univariate ReLU neural networks with a single hidden layer in a binary classification setting. We show that when th…
The Sample Complexity of One-Hidden-Layer Neural Networks
Gal Vardi, Ohad Shamir, Nathan Srebro
We study norm-based uniform convergence bounds for neural networks, aiming at a tight understanding of how these are affected by the architecture and type of norm constraint, for t…
Gradient Methods Provably Converge to Non-Robust Networks
Gal Vardi, Gilad Yehudai, Ohad Shamir
Despite a great deal of research, it is still unclear why neural networks are so susceptible to adversarial examples. In this work, we identify natural settings where depth- ReL…
Width is Less Important than Depth in ReLU Neural Networks
Gal Vardi, Gilad Yehudai, Ohad Shamir
We solve an open question from Lu et al. (2017), by showing that any target network with inputs in can be approximated by a width network (independent of the…
Implicit Regularization Towards Rank Minimization in ReLU Networks
Nadav Timor, Gal Vardi, Ohad Shamir
We study the conjectured relationship between the implicit regularization in neural networks, trained with gradient-based methods, and rank minimization of their weight matrices. P…