Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Abstract

Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as well as the prediction accuracy. Coarse-grained pruning creates regular sparsity patterns, making it more amenable for hardware acceleration but more challenging to maintain the same accuracy. In this paper we quantitatively measure the trade-off between sparsity regularity and prediction accuracy, providing insights in how to maintain accuracy while having more a more structured sparsity pattern. Our experimental results show that coarse-grained pruning can achieve a sparsity ratio similar to unstructured pruning without loss of accuracy. Moreover, due to the index saving effect, coarse-grained pruning is able to obtain a better compression ratio than fine-grained sparsity at the same accuracy threshold. Based on the recent sparse convolutional neural network accelerator (SCNN), our experiments further demonstrate that coarse-grained sparsity saves about 2x the memory references compared to fine-grained sparsity. Since memory reference is more than two orders of magnitude more expensive than arithmetic operations, the regularity of sparse structure leads to more efficient hardware design.

submitted to NIPS 2017

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Densely Connected Convolutional Networks
• Pruning Filters for Efficient ConvNets
• Learning Structured Sparsity in Deep Neural Networks
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• LCNN: Lookup-based Convolutional Neural Network

Cited by in corpus (59)

• A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
• Block-Sparse Recurrent Neural Networks
• Accelerator-Aware Pruning for Convolutional Neural Networks
• Balanced Sparsity for Efficient DNN Inference on GPU
• A novel channel pruning method for deep neural network compression
• TinyVers: A Tiny Versatile System-on-chip with State-Retentive eMRAM for ML Inference at the Extreme Edge
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• Progressive Weight Pruning of Deep Neural Networks using ADMM
• Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks
• HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
• StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs
• Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
• Efficient Quantized Sparse Matrix Operations on Tensor Cores
• Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM
• MARS: Multi-macro Architecture SRAM CIM-Based Accelerator with Co-designed Compressed Neural Networks
• PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices
• Escoin: Efficient Sparse Convolutional Neural Network Inference on GPUs
• ZeroQ: A Novel Zero Shot Quantization Framework
• HAWQV3: Dyadic Neural Network Quantization
• Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks
• Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition
• I-BERT: Integer-only BERT Quantization
• Copernicus: Characterizing the Performance Implications of Compression Formats Used in Sparse Workloads
• CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way
• ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers
• Sparse Weight Activation Training
• Standing on the Shoulders of Giants: Hardware and Neural Architecture Co-Search with Hot Start
• Depthwise Multiception Convolution for Reducing Network Parameters without Sacrificing Accuracy
• BRDS: An FPGA-based LSTM Accelerator with Row-Balanced Dual-Ratio Sparsification
• Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression
• VSCNN: Convolution Neural Network Accelerator With Vector Sparsity
• Accelerating CNN Training by Pruning Activation Gradients
• An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices
• Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices
• Efficient Hardware Realization of Convolutional Neural Networks using Intra-Kernel Regular Pruning
• A Pre-defined Sparse Kernel Based Convolution for Deep CNNs
• NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
• Hessian-Aware Pruning and Optimal Neural Implant
• FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays
• PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution
• A Unified DNN Weight Compression Framework Using Reweighted Optimization Methods
• High Area/Energy Efficiency RRAM CNN Accelerator with Kernel-Reordering Weight Mapping Scheme Based on Pattern Pruning
• Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
• Graph-Adaptive Pruning for Efficient Inference of Convolutional Neural Networks
• Structured Compression by Weight Encryption for Unstructured Pruning and Quantization
• CompactNet: Platform-Aware Automatic Optimization for Convolutional Neural Networks
• Cross-Channel Intragroup Sparsity Neural Network
• Channel-wise pruning of neural networks with tapering resource constraint
• How Does BN Increase Collapsed Neural Network Filters?
• Efficient Structured Pruning and Architecture Searching for Group Convolution
• RingCNN: Exploiting Algebraically-Sparse Ring Tensors for Energy-Efficient CNN-Based Computational Imaging
• Self-grouping Convolutional Neural Networks
• Cascade Weight Shedding in Deep Neural Networks: Benefits and Pitfalls for Network Pruning
• SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
• Architecture-aware Network Pruning for Vision Quality Applications
• Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
• Structured in Space, Randomized in Time: Leveraging Dropout in RNNs for Efficient Training
• MICIK: MIning Cross-Layer Inherent Similarity Knowledge for Deep Model Compression
• Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimization