clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions
arXiv:1712.06145
Abstract
Depthwise convolution and grouped convolution has been successfully applied to improve the efficiency of convolutional neural network (CNN). We suggest that these models can be considered as special cases of a generalized convolution operation, named channel local convolution(CLC), where an output channel is computed using a subset of the input channels. This definition entails computation dependency relations between input and output channels, which can be represented by a channel dependency graph(CDG). By modifying the CDG of grouped convolution, a new CLC kernel named interlaced grouped convolution (IGC) is created. Stacking IGC and GC kernels results in a convolution block (named CLC Block) for approximating regular convolution. By resorting to the CDG as an analysis tool, we derive the rule for setting the meta-parameters of IGC and GC and the framework for minimizing the computational cost. A new CNN model named clcNet is then constructed using CLC blocks, which shows significantly higher computational efficiency and fewer parameters compared to state-of-the-art networks, when being tested using the ImageNet-1K dataset. Source code is available at https://github.com/dqzhang17/clcnet.torch .
References in corpus (13)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Wide Residual Networks
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Speeding up Convolutional Neural Networks with Low Rank Expansions
- Aggregated Residual Transformations for Deep Neural Networks
- Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
- Fast Training of Convolutional Networks through FFTs
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
- Fast Convolutional Nets With fbfft: A GPU Performance Evaluation