Phantom: A High-Performance Computational Core for Sparse Convolutional Neural Networks

Abstract

Sparse convolutional neural networks (CNNs) have gained significant traction over the past few years as sparse CNNs can drastically decrease the model size and computations, if exploited befittingly, as compared to their dense counterparts. Sparse CNNs often introduce variations in the layer shapes and sizes, which can prevent dense accelerators from performing well on sparse CNN models. Recently proposed sparse accelerators like SCNN, Eyeriss v2, and SparTen, actively exploit the two-sided or full sparsity, that is, sparsity in both weights and activations, for performance gains. These accelerators, however, either have inefficient micro-architecture, which limits their performance, have no support for non-unit stride convolutions and fully-connected (FC) layers, or suffer massively from systematic load imbalance. To circumvent these issues and support both sparse and dense models, we propose Phantom, a multi-threaded, dynamic, and flexible neural computational core. Phantom uses sparse binary mask representation to actively lookahead into sparse computations, and dynamically schedule its computational threads to maximize the thread utilization and throughput. We also generate a two-dimensional (2D) mesh architecture of Phantom neural computational cores, which we refer to as Phantom-2D accelerator, and propose a novel dataflow that supports all layers of a CNN, including unit and non-unit stride convolutions, and FC layers. In addition, Phantom-2D uses a two-level load balancing strategy to minimize the computational idling, thereby, further improving the hardware utilization. To show support for different types of layers, we evaluate the performance of the Phantom architecture on VGG16 and MobileNet. Our simulations show that the Phantom-2D accelerator attains a performance gain of 12x, 4.1x, 1.98x, and 2.36x, over dense architectures, SCNN, SparTen, and Eyeriss v2, respectively.

arXiv commentA version of this work is currently under review at the ACM Transactions on Embedded Computing Systems (TECS)

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Natural Language Processing (almost) from Scratch
• Deep Residual Learning for Image Recognition
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• EfficientNetV2: Smaller Models and Faster Training
• Deep Learning with Limited Numerical Precision
• Fixed Point Quantization of Deep Convolutional Networks
• Convolutional Neural Networks using Logarithmic Data Representation
• SpArch: Efficient Architecture for Sparse Matrix Multiplication
• Machine Learning Based Cyber Attacks Targeting on Controlled Information: A Survey
• VWA: Hardware Efficient Vectorwise Accelerator for Convolutional Neural Network
• PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
• Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How