Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?

Abstract

Large deep neural network (DNN) models pose the key challenge to energy efficiency due to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or SRAM operations. It motivates the intensive research on model compression with two main approaches. Weight pruning leverages the redundancy in the number of weights and can be performed in a non-structured, which has higher flexibility and pruning rate but incurs index accesses due to irregular weights, or structured manner, which preserves the full matrix structure with lower pruning rate. Weight quantization leverages the redundancy in the number of bits in weights. Compared to pruning, quantization is much more hardware-friendly, and has become a "must-do" step for FPGA and ASIC implementations. This paper provides a definitive answer to the question for the first time. First, we build ADMM-NN-S by extending and enhancing ADMM-NN, a recently proposed joint weight pruning and quantization framework. Second, we develop a methodology for fair and fundamental comparison of non-structured and structured pruning in terms of both storage and computation efficiency. Our results show that ADMM-NN-S consistently outperforms the prior art: (i) it achieves 348x, 36x, and 8x overall weight pruning on LeNet-5, AlexNet, and ResNet-50, respectively, with (almost) zero accuracy loss; (ii) we demonstrate the first fully binarized (for all layers) DNNs can be lossless in accuracy in many cases. These results provide a strong baseline and credibility of our study. Based on the proposed comparison framework, with the same accuracy and quantization, the results show that non-structrued pruning is not competitive in terms of both storage and computation efficiency. Thus, we conclude that non-structured pruning is considered harmful. We urge the community not to continue the DNN inference acceleration for non-structured sparsity.

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• How transferable are features in deep neural networks?
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• cuDNN: Efficient Primitives for Deep Learning
• Rethinking the Value of Network Pruning
• Binarized Neural Networks
• Pruning Filters for Efficient ConvNets
• Fixed Point Quantization of Deep Convolutional Networks
• A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
• Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
• Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
• Bag of Tricks for Image Classification with Convolutional Neural Networks
• VIBNN: Hardware Acceleration of Bayesian Neural Networks
• 2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy
• Differentiable Fine-grained Quantization for Deep Neural Network Compression
• Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation

Cited by in corpus (6)

• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• Rethinking Weight Decay For Efficient Neural Network Pruning
• Teachers Do More Than Teach: Compressing Image-to-Image Models
• Towards Latency-aware DNN Optimization with GPU Runtime Analysis and Tail Effect Elimination
• AntiDote: Attention-based Dynamic Optimization for Neural Network Runtime Efficiency
• Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI