Powerpropagation: A sparsity inducing weight reparameterisation

Jonathan Schwarz, Siddhant M. Jayakumar, Razvan Pascanu +2

The training of sparse neural networks is becoming an increasingly important tool for reducing the computational footprint of models at training and evaluation, as well enabling th…