Redundant representations help generalization in wide neural networks

Diego Doimo, Aldo Glielmo, Sebastian Goldt +1

Deep neural networks (DNNs) defy the classical bias-variance trade-off: adding parameters to a DNN that interpolates its training data will typically improve its generalization per…