Structured Pruning for Efficient ConvNets via Incremental Regularization

Abstract

Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning. To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods.

Accepted by NIPS 2018 workshop on "Compact Deep Neural Network Representation with Industrial Applications"

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Improving neural networks by preventing co-adaptation of feature detectors
• Caffe: Convolutional Architecture for Fast Feature Embedding
• How transferable are features in deep neural networks?
• cuDNN: Efficient Primitives for Deep Learning
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Channel Pruning for Accelerating Very Deep Neural Networks
• Compact Deep Convolutional Neural Networks With Coarse Pruning

Cited by in corpus (3)

• Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness
• Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
• Directed-Weighting Group Lasso for Eltwise Blocked CNN Pruning