EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning

Abstract

Finding out the computational redundant part of a trained Deep Neural Network (DNN) is the key question that pruning algorithms target on. Many algorithms try to predict model performance of the pruned sub-nets by introducing various evaluation methods. But they are either inaccurate or very complicated for general application. In this work, we present a pruning method called EagleEye, in which a simple yet efficient evaluation component based on adaptive batch normalization is applied to unveil a strong correlation between different pruned DNN structures and their final settled accuracy. This strong correlation allows us to fast spot the pruned candidates with highest potential accuracy without actually fine-tuning them. This module is also general to plug-in and improve some existing pruning algorithms. EagleEye achieves better pruning performance than all of the studied pruning algorithms in our experiments. Concretely, to prune MobileNet V1 and ResNet-50, EagleEye outperforms all compared methods by up to 3.8%. Even in the more challenging experiments of pruning the compact model of MobileNet V1, EagleEye achieves the highest accuracy of 70.9% with an overall 50% operations (FLOPs) pruned. All accuracy results are Top-1 ImageNet classification accuracy. Source code and models are accessible to open-source community https://github.com/anonymous47823493/EagleEye .

Accepted in ECCV 2020(Oral). Codes are available on https://github.com/anonymous47823493/EagleEye

References in corpus (11)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Pruning Filters for Efficient ConvNets
• Revisiting Batch Normalization For Practical Domain Adaptation
• AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
• Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
• MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
• Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
• Building Efficient ConvNets using Redundant Feature Pruning
• AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates
• Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

Cited by in corpus (4)

• End-to-End Supermask Pruning: Learning to Prune Image Captioning Models
• Exploring Gradient Flow Based Saliency for DNN Model Compression
• Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion
• Network Pruning using Adaptive Exemplar Filters