Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference

Rishav Mukherji, Mark Schöne, Khaleelulla Khan Nazeer +2

Artificial neural networks open up unprecedented machine learning capabilities at the cost of ever growing computational requirements. Sparsifying the parameters, often achieved th…