Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations

Abstract

Deep convolutional neural network (CNN) inference requires significant amount of memory and computation, which limits its deployment on embedded devices. To alleviate these problems to some extent, prior research utilize low precision fixed-point numbers to represent the CNN weights and activations. However, the minimum required data precision of fixed-point weights varies across different networks and also across different layers of the same network. In this work, we propose using floating-point numbers for representing the weights and fixed-point numbers for representing the activations. We show that using floating-point representation for weights is more efficient than fixed-point representation for the same bit-width and demonstrate it on popular large-scale CNNs such as AlexNet, SqueezeNet, GoogLeNet and VGG-16. We also show that such a representation scheme enables compact hardware multiply-and-accumulate (MAC) unit design. Experimental results show that the proposed scheme reduces the weight storage by up to 36% and power consumption of the hardware multiplier by up to 50%.

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Deep Learning with Limited Numerical Precision
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• Hardware-oriented Approximation of Convolutional Neural Networks
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
• Reduced-Precision Strategies for Bounded Memory in Deep Neural Nets
• Accelerating Deep Convolutional Networks using low-precision and sparsity

Cited by in corpus (17)

• Hello Edge: Keyword Spotting on Microcontrollers
• Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going
• Trading-off Accuracy and Energy of Deep Inference on Embedded Systems: A Co-Design Approach
• Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
• Quantization of Deep Neural Networks for Accumulator-constrained Processors
• Relaxed Quantization for Discretized Neural Networks
• EffNet: An Efficient Structure for Convolutional Neural Networks
• Rethinking Machine Learning Development and Deployment for Edge Devices
• Bi-Real Net: Binarizing Deep Network Towards Real-Network Performance
• Phoenix: A Low-Precision Floating-Point Quantization Oriented Architecture for Convolutional Neural Networks
• Digital Neuron: A Hardware Inference Accelerator for Convolutional Deep Neural Networks
• Enabling Binary Neural Network Training on the Edge
• Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagation
• CloudifierNet -- Deep Vision Models for Artificial Image Processing
• Low Precision Floating-point Arithmetic for High Performance FPGA-based CNN Acceleration
• KML: Using Machine Learning to Improve Storage Systems
• Multi-Task Network Pruning and Embedded Optimization for Real-time Deployment in ADAS