CAT: Compression-Aware Training for bandwidth reduction

Abstract

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving visual processing tasks. One of the major obstacles hindering the ubiquitous use of CNNs for inference is their relatively high memory bandwidth requirements, which can be a main energy consumer and throughput bottleneck in hardware accelerators. Accordingly, an efficient feature map compression method can result in substantial performance gains. Inspired by quantization-aware training approaches, we propose a compression-aware training (CAT) method that involves training the model in a way that allows better compression of feature maps during inference. Our method trains the model to achieve low-entropy feature maps, which enables efficient compression at inference time using classical transform coding methods. CAT significantly improves the state-of-the-art results reported for quantization. For example, on ResNet-34 we achieve 73.1% accuracy (0.2% degradation from the baseline) with an average representation of only 1.79 bits per value. Reference implementation accompanies the paper at https://github.com/CAT-teams/CAT

References in corpus (13)

• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• The State of Sparsity in Deep Neural Networks
• Sparse Networks from Scratch: Faster Training without Losing Performance
• Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
• NICE: Noise Injection and Clamping Estimation for Neural Network Quantization
• HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
• Feature Map Transform Coding for Energy-Efficient CNN Inference
• Quantization for Rapid Deployment of Deep Neural Networks
• LUTNet: Rethinking Inference in FPGA Soft Logic
• Compressibility Loss for Neural Network Weights
• RBCN: Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

Cited by in corpus (4)

• ZeroQ: A Novel Zero Shot Quantization Framework
• HCM: Hardware-Aware Complexity Metric for Neural Network Architectures
• A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
• Hybrid and Non-Uniform quantization methods using retro synthesis data for efficient inference