536 citations · 4.4k across the 97 of their papers we have counts for
8 papers · 1 filter
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks
Itay Safran, Ohad Shamir
We consider the optimization problem associated with training simple ReLU neural networks of the form with…
Size-Independent Sample Complexity of Neural Networks
Noah Golowich, Alexander Rakhlin, Ohad Shamir
We study the sample complexity of learning neural networks, by providing new bounds on their Rademacher complexity assuming norm constraints on the parameter matrix of each layer.…
Weight Sharing is Crucial to Succesful Optimization
Shai Shalev-Shwartz, Ohad Shamir, Shaked Shammah
Exploiting the great expressive power of Deep Neural Network architectures, relies on the ability to train them. While current theoretical work provides, mostly, results showing th…
Oracle Complexity of Second-Order Methods for Smooth Convex Optimization
Yossi Arjevani, Ohad Shamir, Ron Shiff
Second-order methods, which utilize gradients as well as Hessians to optimize a given function, are of major importance in mathematical optimization. In this work, we prove tight b…
Bandit Regret Scaling with the Effective Loss Range
Nicolò Cesa-Bianchi, Ohad Shamir
We study how the regret guarantees of nonstochastic multi-armed bandits can be improved, if the effective range of the losses in each round is small (e.g. the maximal difference be…
Failures of Gradient-Based Deep Learning
Shai Shalev-Shwartz, Ohad Shamir, Shaked Shammah
In recent years, Deep Learning has become the go-to solution for a broad range of applications, often outperforming state-of-the-art. However, it is important, for both theoreticia…