536 citations · 4.4k across the 97 of their papers we have counts for
85 papers · 1 filter
Logarithmic Width Suffices for Robust Memorization
Amitsour Egosi, Gilad Yehudai, Ohad Shamir
The memorization capacity of neural networks with a given architecture has been thoroughly studied in many works. Specifically, it is well-known that memorizing samples can be…
Depth Separation in Norm-Bounded Infinite-Width Neural Networks
Suzanna Parkinson, Greg Ongie, Rebecca Willett +2
We study depth separation in infinite-width neural networks, where complexity is controlled by the overall squared -norm of the weights (sum of squares of all weights in th…
Generalization in Kernel Regression Under Realistic Assumptions
Daniel Barzilai, Ohad Shamir
It is by now well-established that modern over-parameterized models seem to elude the bias-variance tradeoff and generalize well despite overfitting noise. Many recent works attemp…
Initialization-Dependent Sample Complexity of Linear Predictors and Neural Networks
Roey Magen, Ohad Shamir
We provide several new results on the sample complexity of vector-valued linear predictors (parameterized by a matrix), and more generally neural networks. Focusing on size-indepen…
From Tempered to Benign Overfitting in ReLU Neural Networks
Guy Kornowski, Gilad Yehudai, Ohad Shamir
Overparameterized neural networks (NNs) are observed to generalize well even when trained to perfectly fit noisy data. This phenomenon motivated a large body of work on "benign ove…
Deterministic Nonsmooth Nonconvex Optimization
Michael I. Jordan, Guy Kornowski, Tianyi Lin +2
We study the complexity of optimizing nonsmooth nonconvex Lipschitz functions by producing -stationary points. Several recent works have presented randomized algorithms that…