IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

Abstract

In this paper, we are interested in building lightweight and efficient convolutional neural networks. Inspired by the success of two design patterns, composition of structured sparse kernels, e.g., interleaved group convolutions (IGC), and composition of low-rank kernels, e.g., bottle-neck modules, we study the combination of such two design patterns, using the composition of structured sparse low-rank kernels, to form a convolutional kernel. Rather than introducing a complementary condition over channels, we introduce a loose complementary condition, which is formulated by imposing the complementary condition over super-channels, to guide the design for generating a dense convolutional kernel. The resulting network is called IGCV3. We empirically demonstrate that the combination of low-rank and sparse kernels boosts the performance and the superiority of our proposed approach to the state-of-the-arts, IGCV2 and MobileNetV2 over image classification on CIFAR and ImageNet and object detection on COCO.

arXiv comment10 pages, 2 figures, accepted by BMVC 2018

References in corpus (19)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Striving for Simplicity: The All Convolutional Net
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Densely Connected Convolutional Networks
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Training Very Deep Networks
• Ternary Weight Networks
• Trained Ternary Quantization
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Learning Structured Sparsity in Deep Neural Networks
• Xception: Deep Learning with Depthwise Separable Convolutions
• Channel Pruning for Accelerating Very Deep Neural Networks
• Deep Convolutional Neural Networks with Merge-and-Run Mappings

Cited by in corpus (12)

• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• DyNet: Dynamic Convolution for Accelerating Convolutional Neural Networks
• VarGNet: Variable Group Convolutional Neural Network for Efficient Embedded Computing
• Rethinking Bottleneck Structure for Efficient Mobile Network Design
• Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search
• XSepConv: Extremely Separated Convolution
• FeatherNets: Convolutional Neural Networks as Light as Feather for Face Anti-spoofing
• Building Efficient Deep Neural Networks with Unitary Group Convolutions
• SeesawFaceNets: sparse and robust face verification model for mobile platform
• Cross-Channel Intragroup Sparsity Neural Network
• Self-grouping Convolutional Neural Networks
• Comb Convolution for Efficient Convolutional Architecture