Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing

Abstract

We discuss several modifications and extensions over the previous proposed Cnvlutin (CNV) accelerator for convolutional and fully-connected layers of Deep Learning Network. We first describe different encodings of the activations that are deemed ineffectual. The encodings have different memory overhead and energy characteristics. We propose using a level of indirection when accessing activations from memory to reduce their memory footprint by storing only the effectual activations. We also present a modified organization that detects the activations that are deemed as ineffectual while fetching them from memory. This is different than the original design that instead detected them at the output of the preceding layer. Finally, we present an extended CNV that can also skip ineffectual weights.

arXiv comment6 pages, 5 figures

References in corpus (1)

• Reduced-Precision Strategies for Bounded Memory in Deep Neural Nets

Cited by in corpus (8)

• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training and Inference
• Pruning and Quantization for Deep Neural Network Acceleration: A Survey
• SBNet: Sparse Blocks Network for Fast Inference
• Full deep neural network training on a pruned weight budget
• Self-Supervised GAN Compression
• Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Driving
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization