Network Pruning using Adaptive Exemplar Filters

Abstract

Popular network pruning algorithms reduce redundant information by optimizing hand-crafted models, and may cause suboptimal performance and long time in selecting filters. We innovatively introduce adaptive exemplar filters to simplify the algorithm design, resulting in an automatic and efficient pruning approach called EPruner. Inspired by the face recognition community, we use a message passing algorithm Affinity Propagation on the weight matrices to obtain an adaptive number of exemplars, which then act as the preserved filters. EPruner breaks the dependency on the training data in determining the "important" filters and allows the CPU implementation in seconds, an order of magnitude faster than GPU based SOTAs. Moreover, we show that the weights of exemplars provide a better initialization for the fine-tuning. On VGGNet-16, EPruner achieves a 76.34%-FLOPs reduction by removing 88.80% parameters, with 0.06% accuracy improvement on CIFAR-10. In ResNet-152, EPruner achieves a 65.12%-FLOPs reduction by removing 64.18% parameters, with only 0.71% top-5 accuracy loss on ILSVRC-2012. Our code can be available at https://github.com/lmbxmu/EPruner.

arXiv commentAccepted by IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS)

References in corpus (12)

• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Rethinking the Value of Network Pruning
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• On the importance of single directions for generalization
• Global Sparse Momentum SGD for Pruning Very Deep Neural Networks
• MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
• Revisiting the Importance of Individual Units in CNNs via Ablation
• Rotated Binary Neural Network
• Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
• EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning