39 citations · 139 across the 16 of their papers we have counts for
20 papers
Depth Separation in Norm-Bounded Infinite-Width Neural Networks
Suzanna Parkinson, Greg Ongie, Rebecca Willett +2
We study depth separation in infinite-width neural networks, where complexity is controlled by the overall squared -norm of the weights (sum of squares of all weights in th…
How do Minimum-Norm Shallow Denoisers Look in Function Space?
Chen Zeno, Greg Ongie, Yaniv Blumenfeld +2
Neural network (NN) denoisers are an essential building block in many common tasks, ranging from image reconstruction to image generation. However, the success of these models is n…
The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks
Mor Shpigel Nacson, Rotem Mulayoff, Greg Ongie +2
We study the type of solutions to which stochastic gradient descent converges when used to train a single hidden-layer multivariate ReLU network with the quadratic loss. Our result…
ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Models
Suzanna Parkinson, Greg Ongie, Rebecca Willett
Neural networks often operate in the overparameterized regime, in which there are far more parameters than training samples, allowing the training data to be fit perfectly. That is…
The Role of Linear Layers in Nonlinear Interpolating Networks
Greg Ongie, Rebecca Willett
This paper explores the implicit bias of overparameterized neural networks of depth greater than two layers. Our framework considers a family of networks of varying depth that all…
Deep Equilibrium Architectures for Inverse Problems in Imaging
Davis Gilton, Gregory Ongie, Rebecca Willett
Recent efforts on solving inverse problems in imaging via deep neural networks use architectures inspired by a fixed number of iterations of an optimization method. The number of i…