More is Less: Inducing Sparsity via Overparameterization

Hung-Hsu Chou, Johannes Maly, Holger Rauhut

In deep learning it is common to overparameterize neural networks, that is, to use more parameters than training samples. Quite surprisingly training the neural network via (stocha…