Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Abstract

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Based on tutorial on DNN Hardware at eyeriss.mit.edu/tutorial.html

References in corpus (17)

• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Natural Language Processing (almost) from Scratch
• Caffe: Convolutional Architecture for Fast Feature Embedding
• WaveNet: A Generative Model for Raw Audio
• Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• cuDNN: Efficient Primitives for Deep Learning
• Trained Ternary Quantization
• Compressing Neural Networks with the Hashing Trick
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Learning Structured Sparsity in Deep Neural Networks
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Fractional Max-Pooling
• Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
• Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks

Cited by in corpus (10)

• Marvel: A Data-centric Compiler for DNN Operators on Spatial Accelerators
• Serpens: A High Bandwidth Memory Based Accelerator for General-Purpose Sparse Matrix-Vector Multiplication
• Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression
• Using Deep Learning and Explainable Artificial Intelligence in Patients' Choices of Hospital Levels
• Group Pruning using a Bounded-Lp norm for Group Gating and Regularization
• Energy Predictive Models for Convolutional Neural Networks on Mobile Platforms
• Power-Based Attacks on Spatial DNN Accelerators
• Memory System Designed for Multiply-Accumulate (MAC) Engine Based on Stochastic Computing