C3: Concentrated-Comprehensive Convolution and its application to semantic segmentation

Abstract

One of the practical choices for making a lightweight semantic segmentation model is to combine a depth-wise separable convolution with a dilated convolution. However, the simple combination of these two methods results in an over-simplified operation which causes severe performance degradation due to loss of information contained in the feature map. To resolve this problem, we propose a new block called Concentrated-Comprehensive Convolution (C3) which applies the asymmetric convolutions before the depth-wise separable dilated convolution to compensate for the information loss due to dilated convolution. The C3 block consists of a concentration stage and a comprehensive convolution stage. The first stage uses two depth-wise asymmetric convolutions for compressed information from the neighboring pixels to alleviate the information loss. The second stage increases the receptive field by using a depth-wise separable dilated convolution from the feature map of the first stage. We applied the C3 block to various segmentation frameworks (ESPNet, DRN, ERFNet, ENet) for proving the beneficial properties of our proposed method. Experimental results show that the proposed method preserves the original accuracies on Cityscapes dataset while reducing the complexity. Furthermore, we modified ESPNet to achieve about 2% better performance while reducing the number of parameters by half and the number of FLOPs by 35% compared with the original ESPNet. Finally, experiments on ImageNet classification task show that C3 block can successfully replace dilated convolutions.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time

Cited by in corpus (6)

• LiteSeg: A Novel Lightweight ConvNet for Semantic Segmentation
• ExtremeC3Net: Extreme Lightweight Portrait Segmentation Networks using Advanced C3-modules
• HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers
• SINet: Extreme Lightweight Portrait Segmentation Networks with Spatial Squeeze Modules and Information Blocking Decoder
• Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation
• NoFADE: Analyzing Diminishing Returns on CO2 Investment