Post-Training Quantization for Vision Transformer
arXiv:2106.14156
Abstract
Recently, transformer has achieved remarkable performance on a variety of computer vision applications. Compared with mainstream convolutional neural networks, vision transformers are often of sophisticated architectures for extracting powerful feature representations, which are more difficult to be developed on mobile devices. In this paper, we present an effective post-training quantization algorithm for reducing the memory storage and computational costs of vision transformers. Basically, the quantization task can be regarded as finding the optimal low-bit quantization intervals for weights and inputs, respectively. To preserve the functionality of the attention mechanism, we introduce a ranking loss into the conventional quantization objective that aims to keep the relative order of the self-attention results after quantization. Moreover, we thoroughly analyze the relationship between quantization loss of different layers and the feature diversity, and explore a mixed-precision quantization scheme by exploiting the nuclear norm of each attention map and output feature. The effectiveness of the proposed method is verified on several benchmark models and datasets, which outperforms the state-of-the-art post-training quantization algorithms. For instance, we can obtain an 81.29\% top-1 accuracy using DeiT-B model on ImageNet dataset with about 8-bit quantization.
References in corpus (14)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- Transformer in Transformer
- Trained Ternary Quantization
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Q8BERT: Quantized 8Bit BERT
- Resiliency of Deep Neural Networks under Quantization
- Post-training 4-bit quantization of convolution networks for rapid-deployment
- Pre-Trained Image Processing Transformer
- Training with Quantization Noise for Extreme Model Compression
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
- BinaryBERT: Pushing the Limit of BERT Quantization
- Patch Slimming for Efficient Vision Transformers