SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation

Abstract

We present SmartExchange, an algorithm-hardware co-design framework to trade higher-cost memory storage/access for lower-cost computation, for energy-efficient inference of deep neural networks (DNNs). We develop a novel algorithm to enforce a specially favorable DNN weight structure, where each layerwise weight matrix can be stored as the product of a small basis matrix and a large sparse coefficient matrix whose non-zero elements are all power-of-2. To our best knowledge, this algorithm is the first formulation that integrates three mainstream model compression ideas: sparsification or pruning, decomposition, and quantization, into one unified framework. The resulting sparse and readily-quantized DNN thus enjoys greatly reduced energy consumption in data movement as well as weight storage. On top of that, we further design a dedicated accelerator to fully utilize the SmartExchange-enforced weights to improve both energy efficiency and latency performance. Extensive experiments show that 1) on the algorithm level, SmartExchange outperforms state-of-the-art compression techniques, including merely sparsification or pruning, decomposition, and quantization, in various ablation studies based on nine DNN models and four datasets; and 2) on the hardware level, the proposed SmartExchange based accelerator can improve the energy efficiency by up to 6.7$\times$ and the speedup by up to 19.2$\times$ over four state-of-the-art DNN accelerators, when benchmarked on seven DNN models (including four standard DNNs, two compact DNN models, and one segmentation model) and three datasets.

Accepted by 47th International Symposium on Computer Architecture (ISCA'2020)

References in corpus (9)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Compressing Deep Convolutional Networks using Vector Quantization
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• Training Deep Neural Networks with 8-bit Floating Point Numbers
• Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference
• Model Compression with Adversarial Robustness: A Unified Optimization Framework
• Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers

Cited by in corpus (4)

• ShiftAddNet: A Hardware-Inspired Deep Network
• DNA: Differentiable Network-Accelerator Co-Search
• GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework
• RingCNN: Exploiting Algebraically-Sparse Ring Tensors for Energy-Efficient CNN-Based Computational Imaging