Survey and Benchmarking of Machine Learning Accelerators

Abstract

Advances in multicore processors and accelerators have opened the flood gates to greater exploration and application of machine learning techniques to a variety of applications. These advances, along with breakdowns of several trends including Moore's Law, have prompted an explosion of processors and accelerators that promise even greater computational and machine learning capabilities. These processors and accelerators are coming in many forms, from CPUs and GPUs to ASICs, FPGAs, and dataflow accelerators. This paper surveys the current state of these processors and accelerators that have been publicly announced with performance and power consumption numbers. The performance and power values are plotted on a scatter graph and a number of dimensions and observations from the trends on this plot are discussed and analyzed. For instance, there are interesting trends in the plot regarding power consumption, numerical precision, and inference versus training. We then select and benchmark two commercially-available low size, weight, and power (SWaP) accelerators as these processors are the most interesting for embedded and mobile machine learning inference applications that are most applicable to the DoD and other SWaP constrained users. We determine how they actually perform with real-world images and neural network models, compare those results to the reported performance and power consumption values and evaluate them against an Intel CPU that is used in some embedded applications.

arXiv comment9 pages, 3 figures, IEEE-HPEC conference, Waltham, MA, September 24-26, 2019

Cited by in corpus (15)

• Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
• Survey of Machine Learning Accelerators
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• AI and ML Accelerator Survey and Trends
• AI Accelerator Survey and Trends
• Integrated Photonic Tensor Processing Unit for a Matrix Multiply: a Review
• Digital Twins for the Designs of Systems: a Perspective
• Pure Tensor Program Rewriting via Access Patterns (Representation Pearl)
• Explainable AI using expressive Boolean formulas
• Demonstrating Analog Inference on the BrainScaleS-2 Mobile System
• Exploring Deep Neural Networks on Edge TPU
• Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference
• Benchmarking network fabrics for data distributed training of deep neural networks
• Lincoln AI Computing Survey (LAICS) and Trends
• TapirXLA: Embedding Fork-Join Parallelism into the XLA Compiler in TensorFlow Using Tapir