Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

Abstract

Previous works utilized ''smaller-norm-less-important'' criterion to prune filters with smaller norm values in a convolutional neural network. In this paper, we analyze this norm-based criterion and point out that its effectiveness depends on two requirements that are not always met: (1) the norm deviation of the filters should be large; (2) the minimum norm of the filters should be small. To solve this problem, we propose a novel filter pruning method, namely Filter Pruning via Geometric Median (FPGM), to compress the model regardless of those two requirements. Unlike previous methods, FPGM compresses CNN models by pruning filters with redundancy, rather than those with ''relatively less'' importance. When applied to two image classification benchmarks, our method validates its usefulness and strengths. Notably, on CIFAR-10, FPGM reduces more than 52% FLOPs on ResNet-110 with even 2.69% relative accuracy improvement. Moreover, on ILSVRC-2012, FPGM reduces more than 42% FLOPs on ResNet-101 without top-5 accuracy drop, which has advanced the state-of-the-art. Code is publicly available on GitHub: https://github.com/he-y/filter-pruning-geometric-median

arXiv commentAccepted to CVPR 2019 (Oral)

References in corpus (6)

• Distilling the Knowledge in a Neural Network
• Trained Ternary Quantization
• A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
• Convolutional neural networks with low-rank regularization
• Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression
• SCSP: Spectral Clustering Filter Pruning with Soft Self-adaption Manners

Cited by in corpus (7)

• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
• Weight Normalization based Quantization for Deep Neural Network Compression
• Joint Pruning & Quantization for Extremely Sparse Neural Networks
• Cascaded channel pruning using hierarchical self-distillation
• COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning
• Deep network as memory space: complexity, generalization, disentangled representation and interpretability