Bayesian Bits: Unifying Quantization and Pruning

Abstract

We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition of the quantization operation, which sequentially considers doubling the bit width. At each new bit width, the residual error between the full precision value and the previously rounded value is quantized. We then decide whether or not to add this quantized residual error for a higher effective bit width and lower quantization noise. By starting with a power-of-two bit width, this decomposition will always produce hardware-friendly configurations, and through an additional 0-bit option, serves as a unified view of pruning and quantization. Bayesian Bits then introduces learnable stochastic gates, which collectively control the bit width of the given tensor. As a result, we can obtain low bit solutions by performing approximate inference over the gates, with prior distributions that encourage most of them to be switched off. We experimentally validate our proposed method on several benchmark datasets and show that we can learn pruned, mixed precision networks that provide a better trade-off between accuracy and efficiency than their static bit width equivalents.

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DARTS: Differentiable Architecture Search
• Compressing Deep Convolutional Networks using Vector Quantization
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Ternary Weight Networks
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Training Deep Nets with Sublinear Memory Cost
• Bayesian Compression for Deep Learning
• Training and Inference with Integers in Deep Neural Networks
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
• HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
• Mixed Precision DNNs: All you need is a good parametrization
• Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks

Cited by in corpus (13)

• Structured Pruning for Deep Convolutional Neural Networks: A survey
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
• Only Train Once: A One-Shot Neural Network Training And Pruning Framework
• Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization
• Dustin: A 16-Cores Parallel Ultra-Low-Power Cluster with 2b-to-32b Fully Flexible Bit-Precision and Vector Lockstep Execution Mode
• Efficient Quantized Sparse Matrix Operations on Tensor Cores
• HAWQV3: Dyadic Neural Network Quantization
• Structured Bayesian Compression for Deep Neural Networks Based on The Turbo-VBI Approach
• Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations
• On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks
• Adaptive quantization with mixed-precision based on low-cost proxy
• Data-free mixed-precision quantization using novel sensitivity metric
• Understanding and Overcoming the Challenges of Efficient Transformer Quantization