Feature Map Transform Coding for Energy-Efficient CNN Inference

Abstract

Convolutional neural networks (CNNs) achieve state-of-the-art accuracy in a variety of tasks in computer vision and beyond. One of the major obstacles hindering the ubiquitous use of CNNs for inference on low-power edge devices is their high computational complexity and memory bandwidth requirements. The latter often dominates the energy footprint on modern hardware. In this paper, we introduce a lossy transform coding approach, inspired by image and video compression, designed to reduce the memory bandwidth due to the storage of intermediate activation calculation results. Our method does not require fine-tuning the network weights and halves the data transfer volumes to the main memory by compressing feature maps, which are highly correlated, with variable length coding. Our method outperform previous approach in term of the number of bits per value with minor accuracy degradation on ResNet-34 and MobileNetV2. We analyze the performance of our approach on a variety of CNN architectures and demonstrate that FPGA implementation of ResNet-18 with our approach results in a reduction of around 40% in the memory energy footprint, compared to quantized network, with negligible impact on accuracy. When allowing accuracy degradation of up to 2%, the reduction of 60% is achieved. A reference implementation is available at https://github.com/CompressTeam/TransformCodingInference

References in corpus (11)

• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Fixed Point Quantization of Deep Convolutional Networks
• Faster gaze prediction with dense networks and Fisher pruning
• A Quantization-Friendly Separable Convolution for MobileNets
• Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
• HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
• Quantization for Rapid Deployment of Deep Neural Networks
• Regularizing Activation Distribution for Training Binarized Deep Networks
• SQuantizer: Simultaneous Learning for Both Sparse and Low-precision Neural Networks
• LUTNet: Rethinking Inference in FPGA Soft Logic

Cited by in corpus (2)

• HCM: Hardware-Aware Complexity Metric for Neural Network Architectures
• CAT: Compression-Aware Training for bandwidth reduction