Towards thinner convolutional neural networks through Gradually Global Pruning
arXiv:1703.09916
Abstract
Deep network pruning is an effective method to reduce the storage and computation cost of deep neural networks when applying them to resource-limited devices. Among many pruning granularities, neuron level pruning will remove redundant neurons and filters in the model and result in thinner networks. In this paper, we propose a gradually global pruning scheme for neuron level pruning. In each pruning step, a small percent of neurons were selected and dropped across all layers in the model. We also propose a simple method to eliminate the biases in evaluating the importance of neurons to make the scheme feasible. Compared with layer-wise pruning scheme, our scheme avoid the difficulty in determining the redundancy in each layer and is more effective for deep networks. Our scheme would automatically find a thinner sub-network in original network under a given performance.
References in corpus (13)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Deep Residual Learning for Image Recognition
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Compressing Deep Convolutional Networks using Vector Quantization
- Ternary Weight Networks
- Trained Ternary Quantization
- Predicting Parameters in Deep Learning
- Pruning Filters for Efficient ConvNets
- Tensorizing Neural Networks
- Structured Pruning of Deep Convolutional Neural Networks
- Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee