Compressing the Backward Pass of Large-Scale Neural Architectures by Structured Activation Pruning

Daniel Barley, Holger Fröning

The rise of Deep Neural Networks (DNNs) has led to an increase in model size and complexity, straining the memory capacity of GPUs. Sparsity in DNNs, characterized as structural or…