29 citations · 91 across the 43 of their papers we have counts for
10 papers · 1 filter
Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data
Zhiwei Xu, Yutong Wang, Spencer Frei +2
Neural networks trained by gradient descent (GD) have exhibited a number of surprising generalization behaviors. First, they can achieve a perfect fit to noisy training data and st…
Noisy Interpolation Learning with Shallow Univariate ReLU Networks
Nirmit Joshi, Gal Vardi, Nathan Srebro
Understanding how overparameterized neural networks generalize despite perfect interpolation of noisy training data is a fundamental question. Mallinar et. al. 2022 noted that neur…
Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses
Gon Buzaglo, Niv Haim, Gilad Yehudai +4
Memorization of training data is an active research area, yet our understanding of the inner workings of neural networks is still in its infancy. Recently, Haim et al. (2022) propo…
An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression
Lijia Zhou, James B. Simon, Gal Vardi +1
We study the cost of overfitting in noisy kernel ridge regression (KRR), which we define as the ratio between the test error of the interpolating ridgeless model and the test error…
Reconstructing Training Data from Multiclass Neural Networks
Gon Buzaglo, Niv Haim, Gilad Yehudai +2
Reconstructing samples from the training set of trained neural networks is a major privacy concern. Haim et al. (2022) recently showed that it is possible to reconstruct training s…
Most Neural Networks Are Almost Learnable
Amit Daniely, Nathan Srebro, Gal Vardi
We present a PTAS for learning random constant-depth networks. We show that for any fixed and depth , there is a poly-time algorithm that for any distribution on $\sqrt{d}…
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization
Spencer Frei, Gal Vardi, Peter L. Bartlett +1
Linear classifiers and leaky ReLU networks trained by gradient flow on the logistic loss have an implicit bias towards solutions which satisfy the Karush--Kuhn--Tucker (KKT) condit…
The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks
Spencer Frei, Gal Vardi, Peter L. Bartlett +1
In this work, we study the implications of the implicit bias of gradient flow on generalization and adversarial robustness in ReLU networks. We focus on a setting where the data co…
Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspaces
Odelia Melamed, Gilad Yehudai, Gal Vardi
Despite a great deal of research, it is still not well-understood why trained neural networks are highly vulnerable to adversarial examples. In this work we focus on two-layer neur…
Computational Complexity of Learning Neural Networks: Smoothness and Degeneracy
Amit Daniely, Nathan Srebro, Gal Vardi
Understanding when neural networks can be learned efficiently is a fundamental question in learning theory. Existing hardness results suggest that assumptions on both the input dis…