Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks

Abstract

Modern Convolutional Neural Networks (CNNs) are complex, encompassing millions of parameters. Their deployment exerts computational, storage and energy demands, particularly on embedded platforms. Existing approaches to prune or sparsify CNNs require retraining to maintain inference accuracy. Such retraining is not feasible in some contexts. In this paper, we explore the sparsification of CNNs by proposing three model-independent methods. Our methods are applied on-the-fly and require no retraining. We show that the state-of-the-art models' weights can be reduced by up to 73% (compression factor of 3.7x) without incurring more than 5% loss in Top-5 accuracy. Additional fine-tuning gains only 8% in sparsity, which indicates that our fast on-the-fly methods are effective.

arXiv commentExtended Version of Our Accepted Paper in NIPS 2018, CDNNRIA Workshop: (https://nips.cc/Conferences/2018/Schedule?showEvent=10941)- Reviews are available at OpenReview (https://openreview.net/forum?id=rkz1YD0vjm)

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• TensorFlow: A system for large-scale machine learning
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Learning both Weights and Connections for Efficient Neural Networks
• Learning Structured Sparsity in Deep Neural Networks
• Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
• A Survey on Compiler Autotuning using Machine Learning
• A Regularized Framework for Sparse and Structured Neural Attention
• Automatic Tuning of Local Memory Use on GPGPUs

Cited by in corpus (3)

• Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
• Supervised Robustness-preserving Data-free Neural Network Pruning
• Out-of-the-box channel pruned networks