An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices
arXiv:2001.07710
Abstract
Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms. However, most of the pruning techniques are essentially trade-offs between model accuracy and regularity which lead to impaired inference accuracy and limited on-device acceleration performance. To solve the problem, we introduce a new sparsity dimension, namely pattern-based sparsity that comprises pattern and connectivity sparsity, and becoming both highly accurate and hardware friendly. With carefully designed patterns, the proposed pruning unprecedentedly and consistently achieves accuracy enhancement and better feature extraction ability on different DNN structures and datasets, and our pattern-aware pruning framework also achieves pattern library extraction, pattern selection, pattern and connectivity pruning and weight training simultaneously. Our approach on the new pattern-based sparsity naturally fits into compiler optimization for highly efficient DNN execution on mobile platforms. To the best of our knowledge, it is the first time that mobile devices achieve real-time inference for the large-scale DNN models thanks to the unique spatial property of pattern-based sparsity and the help of the code generation capability of compilers.
Paper accepted in the 16th European Conference on Computer Vision (ECCV 2020)
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Axiomatic Attribution for Deep Networks
- Pruning Convolutional Neural Networks for Resource Efficient Inference
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Rethinking the Value of Network Pruning
- Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
- Pruning Filters for Efficient ConvNets
- Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
- AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
- The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning
- Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
- 2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy