Towards Compact ConvNets via Structure-Sparsity Regularized Filter Pruning

Abstract

The success of convolutional neural networks (CNNs) in computer vision applications has been accompanied by a significant increase of computation and memory costs, which prohibits its usage on resource-limited environments such as mobile or embedded devices. To this end, the research of CNN compression has recently become emerging. In this paper, we propose a novel filter pruning scheme, termed structured sparsity regularization (SSR), to simultaneously speedup the computation and reduce the memory overhead of CNNs, which can be well supported by various off-the-shelf deep learning libraries. Concretely, the proposed scheme incorporates two different regularizers of structured sparsity into the original objective function of filter pruning, which fully coordinates the global outputs and local pruning operations to adaptively prune filters. We further propose an Alternative Updating with Lagrange Multipliers (AULM) scheme to efficiently solve its optimization. AULM follows the principle of ADMM and alternates between promoting the structured sparsity of CNNs and optimizing the recognition loss, which leads to a very efficient solver (2.5x to the most recent work that directly solves the group sparsity-based regularization). Moreover, by imposing the structured sparsity, the online inference is extremely memory-light, since the number of filters and the output feature maps are simultaneously reduced. The proposed scheme has been deployed to a variety of state-of-the-art CNN structures including LeNet, AlexNet, VGG, ResNet and GoogLeNet over different datasets. Quantitative results demonstrate that the proposed scheme achieves superior performance over the state-of-the-art methods. We further demonstrate the proposed compression scheme for the task of transfer learning, including domain adaptation and object detection, which also show exciting performance gains over the state-of-the-arts.

IEEE Transactions on Neural Networks and Learning Systems (TNNLS)

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Compressing Deep Convolutional Networks using Vector Quantization
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Fast Training of Convolutional Networks through FFTs
• Fast Convolutional Nets With fbfft: A GPU Performance Evaluation
• A Field Guide to Forward-Backward Splitting with a FASTA Implementation
• Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
• Low-complexity Approximate Convolutional Neural Networks
• Structured Pruning of Deep Convolutional Neural Networks
• Attention-Based Guided Structured Sparsity of Deep Neural Networks

Cited by in corpus (4)

• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
• Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
• Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge