The empirical size of trained neural networks

Kevin K. Chen, Anthony Gamst, Alden Walker

ReLU neural networks define piecewise linear functions of their inputs. However, initializing and training a neural network is very different from fitting a linear spline. In this…