XSepConv: Extremely Separated Convolution

Abstract

Depthwise convolution has gradually become an indispensable operation for modern efficient neural networks and larger kernel sizes ($\ge 5$) have been applied to it recently. In this paper, we propose a novel extremely separated convolutional block (XSepConv), which fuses spatially separable convolutions into depthwise convolution to further reduce both the computational cost and parameter size of large kernels. Furthermore, an extra $2\times 2$ depthwise convolution coupled with improved symmetric padding strategy is employed to compensate for the side effect brought by spatially separable convolutions. XSepConv is designed to be an efficient alternative to vanilla depthwise convolution with large kernel sizes. To verify this, we use XSepConv for the state-of-the-art architecture MobileNetV3-Small and carry out extensive experiments on four highly competitive benchmark datasets (CIFAR-10, CIFAR-100, SVHN and Tiny-ImageNet) to demonstrate that XSepConv can indeed strike a better trade-off between accuracy and efficiency.

References in corpus (7)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• MixConv: Mixed Depthwise Convolutional Kernels
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

Cited by in corpus (1)

• Review: Deep Learning in Electron Microscopy