Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win

Utku Evci, Yani A. Ioannou, Cem Keskin +1

Sparse Neural Networks (NNs) can match the generalization of dense NNs using a fraction of the compute/storage for inference, and also have the potential to enable efficient traini…