Learning Versatile Convolution Filters for Efficient Visual Recognition

Abstract

This paper introduces versatile filters to construct efficient convolutional neural networks that are widely used in various visual recognition tasks. Considering the demands of efficient deep learning techniques running on cost-effective hardware, a number of methods have been developed to learn compact neural networks. Most of these works aim to slim down filters in different ways, \eg,~investigating small, sparse or quantized filters. In contrast, we treat filters from an additive perspective. A series of secondary filters can be derived from a primary filter with the help of binary masks. These secondary filters all inherit in the primary filter without occupying more storage, but once been unfolded in computation they could significantly enhance the capability of the filter by integrating information extracted from different receptive fields. Besides spatial versatile filters, we additionally investigate versatile filters from the channel perspective. Binary masks can be further customized for different primary filters under orthogonal constraints. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters, but require less memory and computation cost.

arXiv commentAccepted by TPAMI. Extended version of NeurIPS 2018 paper

References in corpus (36)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• TensorFlow: A system for large-scale machine learning
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Neural Architecture Search with Reinforcement Learning
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• FitNets: Hints for Thin Deep Nets
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Binarized Neural Networks
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Learning Structured Sparsity in Deep Neural Networks
• Channel Pruning for Accelerating Very Deep Neural Networks
• MixConv: Mixed Depthwise Convolutional Kernels
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• Provable Bounds for Learning Some Deep Representations
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Slimmable Neural Networks
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
• PerforatedCNNs: Acceleration through Elimination of Redundant Convolutions
• MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
• SCOP: Scientific Control for Reliable Neural Network Pruning
• LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
• Training Binary Neural Networks through Learning with Noisy Supervision
• ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions
• Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
• FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
• Learning Implicitly Recurrent CNNs Through Parameter Sharing
• Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
• Constructing Fast Network through Deconstruction of Convolution
• FSNet: Compression of Deep Convolutional Neural Networks by Filter Summary

Cited by in corpus (2)

• GhostNets on Heterogeneous Devices via Cheap Operations
• Self-Distribution Binary Neural Networks