The Role of Linear Layers in Nonlinear Interpolating Networks

Greg Ongie, Rebecca Willett

This paper explores the implicit bias of overparameterized neural networks of depth greater than two layers. Our framework considers a family of networks of varying depth that all…