536 citations · 4.4k across the 97 of their papers we have counts for
7 papers · 1 filter
On the Complexity of Finding Small Subgradients in Nonsmooth Optimization
Guy Kornowski, Ohad Shamir
We study the oracle complexity of producing -stationary points of Lipschitz functions, in the sense proposed by Zhang et al. [2020]. While there exist dimension-free randomi…
Reconstructing Training Data from Trained Neural Networks
Niv Haim, Gal Vardi, Gilad Yehudai +2
Understanding to what extent neural networks memorize training data is an intriguing question with practical and theoretical implications. In this paper we show that in some cases…
The Sample Complexity of One-Hidden-Layer Neural Networks
Gal Vardi, Ohad Shamir, Nathan Srebro
We study norm-based uniform convergence bounds for neural networks, aiming at a tight understanding of how these are affected by the architecture and type of norm constraint, for t…
Gradient Methods Provably Converge to Non-Robust Networks
Gal Vardi, Gilad Yehudai, Ohad Shamir
Despite a great deal of research, it is still unclear why neural networks are so susceptible to adversarial examples. In this work, we identify natural settings where depth- ReL…
Width is Less Important than Depth in ReLU Neural Networks
Gal Vardi, Gilad Yehudai, Ohad Shamir
We solve an open question from Lu et al. (2017), by showing that any target network with inputs in can be approximated by a width network (independent of the…
Implicit Regularization Towards Rank Minimization in ReLU Networks
Nadav Timor, Gal Vardi, Ohad Shamir
We study the conjectured relationship between the implicit regularization in neural networks, trained with gradient-based methods, and rank minimization of their weight matrices. P…