BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction

Abstract

We study the challenging task of neural network quantization without end-to-end retraining, called Post-training Quantization (PTQ). PTQ usually requires a small subset of training data but produces less powerful quantized models than Quantization-Aware Training (QAT). In this work, we propose a novel PTQ framework, dubbed BRECQ, which pushes the limits of bitwidth in PTQ down to INT2 for the first time. BRECQ leverages the basic building blocks in neural networks and reconstructs them one-by-one. In a comprehensive theoretical study of the second-order error, we show that BRECQ achieves a good balance between cross-layer dependency and generalization error. To further employ the power of quantization, the mixed precision technique is incorporated in our framework by approximating the inter-layer and intra-layer sensitivity. Extensive experiments on various handcrafted and searched neural architectures are conducted for both image classification and object detection tasks. And for the first time we prove that, without bells and whistles, PTQ can attain 4-bit ResNet and MobileNetV2 comparable with QAT and enjoy 240 times faster production of quantized models. Codes are available at https://github.com/yhhhli/BRECQ.

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• Neural Architecture Search with Reinforcement Learning
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Model compression via distillation and quantization
• Faster gaze prediction with dense networks and Fisher pruning
• Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
• Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks
• Practical Gauss-Newton Optimisation for Deep Learning

Cited by in corpus (7)

• Towards Efficient Post-training Quantization of Pre-trained Language Models
• OMPQ: Orthogonal Mixed Precision Quantization
• IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization
• Fine-grained Data Distribution Alignment for Post-Training Quantization
• Post-Training Sparsity-Aware Quantization
• PTQ-SL: Exploring the Sub-layerwise Post-training Quantization
• MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing