Building Efficient ConvNets using Redundant Feature Pruning
arXiv:1802.07653
Abstract
This paper presents an efficient technique to prune deep and/or wide convolutional neural network models by eliminating redundant features (or filters). Previous studies have shown that over-sized deep neural network models tend to produce a lot of redundant features that are either shifted version of one another or are very similar and show little or no variations; thus resulting in filtering redundancy. We propose to prune these redundant features along with their connecting feature maps according to their differentiation and based on their relative cosine distances in the feature space, thus yielding smaller network size with reduced inference costs and competitive performance. We empirically show on select models and CIFAR-10 dataset that inference costs can be reduced by 40% for VGG-16, 27% for ResNet-56, and 39% for ResNet-110.
References in corpus (5)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Fast Training of Convolutional Networks through FFTs
- DSD: Dense-Sparse-Dense Training for Deep Neural Networks
- The Power of Sparsity in Convolutional Neural Networks
- Regularizing CNNs with Locally Constrained Decorrelations
Cited by in corpus (9)
- Near-Optimal Sparse Allreduce for Distributed Deep Learning
- EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
- A Survey of Visual Analytics Techniques for Machine Learning
- Efficient architecture for deep neural networks with heterogeneous sensitivity
- Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting
- GHFP: Gradually Hard Filter Pruning
- How Compact?: Assessing Compactness of Representations through Layer-Wise Pruning
- Diversity Regularized Adversarial Learning
- Effective Model Compression via Stage-wise Pruning