Compact Deep Convolutional Neural Networks With Coarse Pruning

Abstract

The learning capability of a neural network improves with increasing depth at higher computational costs. Wider layers with dense kernel connectivity patterns furhter increase this cost and may hinder real-time inference. We propose feature map and kernel level pruning for reducing the computational complexity of a deep convolutional neural network. Pruning feature maps reduces the width of a layer and hence does not need any sparse representation. Further, kernel pruning converts the dense connectivity pattern into a sparse one. Due to coarse nature, these pruning granularities can be exploited by GPUs and VLSI based implementations. We propose a simple and generic strategy to choose the least adversarial pruning masks for both granularities. The pruned networks are retrained which compensates the loss in accuracy. We obtain the best pruning ratios when we prune a network with both granularities. Experiments with the CIFAR-10 dataset show that more than 85% sparsity can be induced in the convolution layers with less than 1% increase in the missclassification rate of the baseline network.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Pruning Filters for Efficient ConvNets
• Fast Training of Convolutional Networks through FFTs
• Memory Bounded Deep Convolutional Networks
• Structured Pruning of Deep Convolutional Neural Networks
• Fast ConvNets Using Group-wise Brain Damage

Cited by in corpus (18)

• Channel Pruning for Accelerating Very Deep Neural Networks
• MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
• Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications
• Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
• NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm
• Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression
• Rethinking Weight Decay For Efficient Neural Network Pruning
• A Programmable Approach to Neural Network Compression
• Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks
• Structured Pruning for Efficient ConvNets via Incremental Regularization
• Efficient Hardware Realization of Convolutional Neural Networks using Intra-Kernel Regular Pruning
• Compressing complex convolutional neural network based on an improved deep compression algorithm
• Synthesis and Pruning as a Dynamic Compression Strategy for Efficient Deep Neural Networks
• Three Dimensional Convolutional Neural Network Pruning with Regularization-Based Method
• Going Beyond Classification Accuracy Metrics in Model Compression
• Quantization Mimic: Towards Very Tiny CNN for Object Detection
• Auto Deep Compression by Reinforcement Learning Based Actor-Critic Structure
• A One-step Pruning-recovery Framework for Acceleration of Convolutional Neural Networks