Learning Filter Basis for Convolutional Neural Network Compression
arXiv:1908.08932
Abstract
Convolutional neural networks (CNNs) based solutions have achieved state-of-the-art performances for many computer vision tasks, including classification and super-resolution of images. Usually the success of these methods comes with a cost of millions of parameters due to stacking deep convolutional layers. Moreover, quite a large number of filters are also used for a single convolutional layer, which exaggerates the parameter burden of current methods. Thus, in this paper, we try to reduce the number of parameters of CNNs by learning a basis of the filters in convolutional layers. For the forward pass, the learned basis is used to approximate the original filters and then used as parameters for the convolutional layers. We validate our proposed solution for multiple CNN architectures on image classification and image super-resolution benchmarks and compare favorably to the existing state-of-the-art in terms of reduction of parameters and preservation of accuracy.
arXiv commentCode is available at https://github.com/ofsoundof/learning_filter_basis. ICCV 2019
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Ternary Weight Networks
- Trained Ternary Quantization
- Compressing Neural Networks with the Hashing Trick
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
- Learning the Number of Neurons in Deep Networks
- Multi-bin Trainable Linear Unit for Fast Image Restoration Networks