Context Encoding for Semantic Segmentation

Abstract

Recent work has made significant progress in improving spatial resolution for pixelwise labeling with Fully Convolutional Network (FCN) framework by employing Dilated/Atrous convolution, utilizing multi-scale features and refining boundaries. In this paper, we explore the impact of global contextual information in semantic segmentation by introducing the Context Encoding Module, which captures the semantic context of scenes and selectively highlights class-dependent featuremaps. The proposed Context Encoding Module significantly improves semantic segmentation results with only marginal extra computation cost over FCN. Our approach has achieved new state-of-the-art results 51.7% mIoU on PASCAL-Context, 85.9% mIoU on PASCAL VOC 2012. Our single model achieves a final score of 0.5567 on ADE20K test set, which surpass the winning entry of COCO-Place Challenge in 2017. In addition, we also explore how the Context Encoding Module can improve the feature representation of relatively shallow networks for the image classification on CIFAR-10 dataset. Our 14 layer network has achieved an error rate of 3.45%, which is comparable with state-of-the-art approaches with over 10 times more layers. The source code for the complete system are publicly available.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

References in corpus (24)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Densely Connected Convolutional Networks
• Wide Residual Networks
• ParseNet: Looking Wider to See Better
• Fully Convolutional Networks for Semantic Segmentation
• A Learned Representation For Artistic Style
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Learning Deconvolution Network for Semantic Segmentation
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Aggregated Residual Transformations for Deep Neural Networks
• Path Aggregation Network for Instance Segmentation
• Pyramid Scene Parsing Network
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Identity Mappings in Deep Residual Networks
• Simultaneous Detection and Segmentation
• Semantic Image Segmentation via Deep Parsing Network
• Dilated Residual Networks
• Bridging Category-level and Instance-level Semantic Image Segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Multi-style Generative Network for Real-time Transfer
• MegDet: A Large Mini-Batch Object Detector
• Material Recognition from Local Appearance in Global Context
• Deep TEN: Texture Encoding Network

Cited by in corpus (35)

• ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
• OCNet: Object Context Network for Scene Parsing
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data
• Pyramid Attention Network for Semantic Segmentation
• K-Net: Towards Unified Image Segmentation
• Bag of Tricks for Image Classification with Convolutional Neural Networks
• CGNet: A Light-weight Context Guided Network for Semantic Segmentation
• Densely Connected Pyramid Dehazing Network
• DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation
• Global Aggregation then Local Distribution in Fully Convolutional Networks
• Strip Pooling: Rethinking Spatial Pooling for Scene Parsing
• Deep Co-supervision and Attention Fusion Strategy for Automatic COVID-19 Lung Infection Segmentation on CT Images
• Empirical curvelet based Fully Convolutional Network for supervised texture image segmentation
• Learning Guided Convolutional Network for Depth Completion
• A Novel Upsampling and Context Convolution for Image Semantic Segmentation
• GTA: Global Temporal Attention for Video Action Understanding
• An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks
• Semantic Correlation Promoted Shape-Variant Context for Segmentation
• Temporally Distributed Networks for Fast Video Semantic Segmentation
• Zero-Shot Semantic Segmentation
• Tree-structured Kronecker Convolutional Network for Semantic Segmentation
• PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation
• Collaborative Boundary-aware Context Encoding Networks for Error Map Prediction
• PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask
• AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsing
• Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation
• Feature-Fused Context-Encoding Network for Neuroanatomy Segmentation
• Attention-based fusion of semantic boundary and non-boundary information to improve semantic segmentation
• Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation
• Scene Parsing via Dense Recurrent Neural Networks with Attentional Selection
• Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks
• Shape-Oriented Convolution Neural Network for Point Cloud Analysis
• RethNet: Object-by-Object Learning for Detecting Facial Skin Problems
• Hierarchical Pyramid Representations for Semantic Segmentation