Interleaved Group Convolutions for Deep Neural Networks

Abstract

In this paper, we present a simple and modularized neural network architecture, named interleaved group convolutional neural networks (IGCNets). The main point lies in a novel building block, a pair of two successive interleaved group convolutions: primary group convolution and secondary group convolution. The two group convolutions are complementary: (i) the convolution on each partition in primary group convolution is a spatial convolution, while on each partition in secondary group convolution, the convolution is a point-wise convolution; (ii) the channels in the same secondary partition come from different primary partitions. We discuss one representative advantage: Wider than a regular convolution with the number of parameters and the computation complexity preserved. We also show that regular convolutions, group convolution with summation fusion, and the Xception block are special cases of interleaved group convolutions. Empirical results over standard benchmarks, CIFAR-$10$, CIFAR-$100$, SVHN and ImageNet demonstrate that our networks are more efficient in using parameters and computation complexity with similar or higher accuracy.

To appear in ICCV 2017

References in corpus (20)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Striving for Simplicity: The All Convolutional Net
• FitNets: Hints for Thin Deep Nets
• Densely Connected Convolutional Networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Wide Residual Networks
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Resnet in Resnet: Generalizing Residual Architectures
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Rethinking the Inception Architecture for Computer Vision
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Learning Structured Sparsity in Deep Neural Networks
• Aggregated Residual Transformations for Deep Neural Networks
• Xception: Deep Learning with Depthwise Separable Convolutions
• Deep Networks with Stochastic Depth
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• Swapout: Learning an ensemble of deep architectures
• Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
• The Power of Sparsity in Convolutional Neural Networks

Cited by in corpus (14)

• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• CondenseNet: An Efficient DenseNet using Learned Group Convolutions
• Competitive Inner-Imaging Squeeze and Excitation for Residual Network
• Seesaw-Net: Convolution Neural Network With Uneven Group Convolution
• Rethink ReLU to Training Better CNNs
• PydMobileNet: Improved Version of MobileNets with Pyramid Depthwise Separable Convolution
• Pyramidal Dense Attention Networks for Lightweight Image Super-Resolution
• WaveletNet: Logarithmic Scale Efficient Convolutional Neural Networks for Edge Devices
• Quantization Mimic: Towards Very Tiny CNN for Object Detection
• ESFNet: Efficient Network for Building Extraction from High-Resolution Aerial Images
• Content-adaptive Representation Learning for Fast Image Super-resolution
• Efficient Structured Pruning and Architecture Searching for Group Convolution
• Predicting population neural activity in the Algonauts challenge using end-to-end trained Siamese networks and group convolutions