Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going

Abstract

Deep neural networks have proven to be particularly effective in visual and audio recognition tasks. Existing models tend to be computationally expensive and memory intensive, however, and so methods for hardware-oriented approximation have become a hot topic. Research has shown that custom hardware-based neural network accelerators can surpass their general-purpose processor equivalents in terms of both throughput and energy efficiency. Application-tailored accelerators, when co-designed with approximation-based network training methods, transform large, dense and computationally expensive networks into small, sparse and hardware-efficient alternatives, increasing the feasibility of network deployment. In this article, we provide a comprehensive evaluation of approximation methods for high-performance network inference along with in-depth discussion of their effectiveness for custom hardware implementation. We also include proposals for future research based on a thorough analysis of current trends. This article represents the first survey providing detailed comparisons of custom hardware accelerators featuring approximation for both convolutional and recurrent neural networks, through which we hope to inspire exciting new developments in the field.

Accepted manuscript uploaded 21/01/19. DOA 15/01/19

References in corpus (27)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Deep Learning with Limited Numerical Precision
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Ternary Weight Networks
• Trained Ternary Quantization
• Pruning Filters for Efficient ConvNets
• Learning both Weights and Connections for Efficient Neural Networks
• Fixed Point Quantization of Deep Convolutional Networks
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Learning Structured Sparsity in Deep Neural Networks
• Towards Accurate Binary Convolutional Neural Network
• Adding Gradient Noise Improves Learning for Very Deep Networks
• CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
• Neural Networks with Few Multiplications
• Conditional Computation in Neural Networks for faster models
• Training Quantized Nets: A Deeper Understanding
• Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
• Ternary Neural Networks with Fine-Grained Quantization
• ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks
• Recurrent Neural Networks With Limited Numerical Precision
• Throughput Optimizations for FPGA-based Deep Neural Network Inference
• Learning to Fly by Crashing
• Learning Compact Recurrent Neural Networks
• Dynamic Capacity Networks

Cited by in corpus (22)

• Survey of Machine Learning Accelerators
• Recurrent Neural Networks: An Embedded Computing Perspective
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
• Taurus: A Data Plane Architecture for Per-Packet ML
• AI Accelerator Survey and Trends
• Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
• Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Models
• A Dimensionality Reduction Approach for Convolutional Neural Networks
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors
• DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression
• Rethinking Arithmetic for Deep Neural Networks
• Spatio-temporal evolution of resistance state in simulated memristive networks
• Digit Stability Inference for Iterative Methods Using Redundant Number Representation
• BinArray: A Scalable Hardware Accelerator for Binary Approximated CNNs
• Verifying Quantized Neural Networks using SMT-Based Model Checking
• Enabling Binary Neural Network Training on the Edge
• Making Logic Learnable With Neural Networks
• C3PU: Cross-Coupling Capacitor Processing Unit Using Analog-Mixed Signal In-Memory Computing for AI Inference