Differentiable Joint Pruning and Quantization for Hardware Efficiency
arXiv:2007.10463
Abstract
We present a differentiable joint pruning and quantization (DJPQ) scheme. We frame neural network compression as a joint gradient-based optimization problem, trading off between model pruning and quantization automatically for hardware efficiency. DJPQ incorporates variational information bottleneck based structured pruning and mixed-bit precision quantization into a single differentiable loss function. In contrast to previous works which consider pruning and quantization separately, our method enables users to find the optimal trade-off between both in a single training procedure. To utilize the method for more efficient hardware inference, we extend DJPQ to integrate structured pruning with power-of-two bit-restricted quantization. We show that DJPQ significantly reduces the number of Bit-Operations (BOPs) for several networks while maintaining the top-1 accuracy of original floating-point models (e.g., 53x BOPs reduction in ResNet18 on ImageNet, 43x in MobileNetV2). Compared to the conventional two-stage approach, which optimizes pruning and quantization independently, our scheme outperforms in terms of both accuracy and BOPs. Even when considering bit-restricted quantization, DJPQ achieves larger compression ratios and better accuracy than the two-stage approach.
arXiv commentAccepted to ECCV 2020
References in corpus (17)
- The information bottleneck method
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Pruning Convolutional Neural Networks for Resource Efficient Inference
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Ternary Weight Networks
- Bayesian Compression for Deep Learning
- Training and Inference with Integers in Deep Neural Networks
- Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
- Faster gaze prediction with dense networks and Fisher pruning
- Compressing Neural Networks using the Variational Information Bottleneck
- MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy?
- Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
- ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks
- A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM
- Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks