FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation

Abstract

Modern approaches for semantic segmentation usually employ dilated convolutions in the backbone to extract high-resolution feature maps, which brings heavy computation complexity and memory footprint. To replace the time and memory consuming dilated convolutions, we propose a novel joint upsampling module named Joint Pyramid Upsampling (JPU) by formulating the task of extracting high-resolution feature maps into a joint upsampling problem. With the proposed JPU, our method reduces the computation complexity by more than three times without performance loss. Experiments show that JPU is superior to other upsampling modules, which can be plugged into many existing approaches to reduce computation complexity and improve performance. By replacing dilated convolutions with the proposed JPU module, our method achieves the state-of-the-art performance in Pascal Context dataset (mIoU of 53.13%) and ADE20K dataset (final score of 0.5584) while running 3 times faster.

Code is available in https://github.com/wuhuikai/FastFCN

References in corpus (7)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• ParseNet: Looking Wider to See Better
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Stacked Deconvolutional Network for Semantic Segmentation
• ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Cited by in corpus (28)

• UIU-Net: U-Net in U-Net for Infrared Small Object Detection
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
• Domain Adaptation Meets Zero-Shot Learning: An Annotation-Efficient Approach to Multi-Modality Medical Image Segmentation
• MMSFormer: Multimodal Transformer for Material and Semantic Segmentation
• Improving Semantic Segmentation via Self-Training
• A Novel Upsampling and Context Convolution for Image Semantic Segmentation
• LULC classification by semantic segmentation of satellite images using FastFCN
• OCTAve: 2D en face Optical Coherence Tomography Angiography Vessel Segmentation in Weakly-Supervised Learning with Locality Augmentation
• See More Than Once -- Kernel-Sharing Atrous Convolution for Semantic Segmentation
• MSE Loss with Outlying Label for Imbalanced Classification
• MaLeFiSenta: Machine Learning for FilamentS Identification and orientation in the ISM
• Deep Convolutional Neural Networks with Spatial Regularization, Volume and Star-shape Priori for Image Segmentation
• Empirical Study of Multi-Task Hourglass Model for Semantic Segmentation Task
• CTNet: Context-based Tandem Network for Semantic Segmentation
• TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency
• Adversarial Multiscale Feature Learning for Overlapping Chromosome Segmentation
• Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsing
• Adaptive Context Encoding Module for Semantic Segmentation
• EfficientFCN: Holistically-guided Decoding for Semantic Segmentation
• Location-aware Upsampling for Semantic Segmentation
• Affordance segmentation of hand-occluded containers from exocentric images
• A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection
• G-SemTMO: Tone Mapping with a Trainable Semantic Graph
• Generalizing Adversarial Examples by AdaBelief Optimizer
• CASSOD-Net: Cascaded and Separable Structures of Dilated Convolution for Embedded Vision Systems and Applications
• GINet: Graph Interaction Network for Scene Parsing
• Intra-class Patch Swap for Self-Distillation
• Integrating Deep Learning and Augmented Reality to Enhance Situational Awareness in Firefighting Environments