3 citations · 3 across the 1 of their papers we have counts for
6 papers
On the Optimal Memorization Power of ReLU Neural Networks
Gal Vardi, Gilad Yehudai, Ohad Shamir
We study the memorization power of feedforward ReLU neural networks. We show that such networks can memorize any points that satisfy a mild separability assumption using $\tild…
Size and Depth Separation in Approximating Benign Functions with Neural Networks
Gal Vardi, Daniel Reichman, Toniann Pitassi +1
When studying the expressive power of neural networks, a main challenge is to understand how the size and depth of the network affect its ability to approximate real functions. How…
From Local Pseudorandom Generators to Hardness of Learning
Amit Daniely, Gal Vardi
We prove hardness-of-learning results under a well-studied assumption on the existence of local pseudorandom generators. As we show, this assumption allows us to surpass the curren…
Implicit Regularization in ReLU Networks with the Square Loss
Gal Vardi, Ohad Shamir
Understanding the implicit regularization (or implicit bias) of gradient descent has recently been a very active research area. However, the implicit regularization in nonlinear ne…
Hardness of Learning Neural Networks with Natural Weights
Amit Daniely, Gal Vardi
Neural networks are nowadays highly successful despite strong hardness results. The existing hardness results focus on the network architecture, and assume that the network's weigh…
Neural Networks with Small Weights and Depth-Separation Barriers
Gal Vardi, Ohad Shamir
In studying the expressiveness of neural networks, an important question is whether there are functions which can only be approximated by sufficiently deep networks, assuming their…