Phoenix: A Low-Precision Floating-Point Quantization Oriented Architecture for Convolutional Neural Networks

Abstract

Convolutional neural networks (CNNs) achieve state-of-the-art performance at the cost of becoming deeper and larger. Although quantization (both fixed-point and floating-point) has proven effective for reducing storage and memory access, two challenges -- 1) accuracy loss caused by quantization without calibration, fine-tuning or re-training for deep CNNs and 2) hardware inefficiency caused by floating-point quantization -- prevent processors from completely leveraging the benefits. In this paper, we propose a low-precision floating-point quantization oriented processor, named Phoenix, to address the above challenges. We primarily have three key observations: 1) 8-bit floating-point quantization incurs less error than 8-bit fixed-point quantization; 2) without using any calibration, fine-tuning or re-training techniques, normalization before quantization further reduces accuracy degradation; 3) 8-bit floating-point multiplier achieves higher hardware efficiency than 8-bit fixed-point multiplier if the full-precision product is applied. Based on these key observations, we propose a normalization-oriented 8-bit floating-point quantization method to reduce storage and memory access with negligible accuracy loss (within 0.5%/0.3% for top-1/top-5 accuracy, respectively). We further design a hardware processor to address the hardware inefficiency caused by floating-point multiplier. Compared with a state-of-the-art accelerator, Phoenix is 3.32x and 7.45x better in performance with the same core area for AlexNet and VGG16, respectively.

14 pages, 18 figures, submitted to TVLSI

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Trained Ternary Quantization
• Fixed Point Quantization of Deep Convolutional Networks
• Training deep neural networks with low precision multiplications
• Training Deep Neural Networks with 8-bit Floating Point Numbers
• Neural Networks with Few Multiplications
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines