Semantic Correlation Promoted Shape-Variant Context for Segmentation

Abstract

Context is essential for semantic segmentation. Due to the diverse shapes of objects and their complex layout in various scene images, the spatial scales and shapes of contexts for different objects have very large variation. It is thus ineffective or inefficient to aggregate various context information from a predefined fixed region. In this work, we propose to generate a scale- and shape-variant semantic mask for each pixel to confine its contextual region. To this end, we first propose a novel paired convolution to infer the semantic correlation of the pair and based on that to generate a shape mask. Using the inferred spatial scope of the contextual region, we propose a shape-variant convolution, of which the receptive field is controlled by the shape mask that varies with the appearance of input. In this way, the proposed network aggregates the context information of a pixel from its semantic-correlated region instead of a predefined fixed region. Furthermore, this work also proposes a labeling denoising model to reduce wrong predictions caused by the noisy low-level features. Without bells and whistles, the proposed segmentation network achieves new state-of-the-arts consistently on the six public segmentation datasets.

CVPR 2019, Oral

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Random Fields as Recurrent Neural Networks
• Multi-Scale Context Aggregation by Dilated Convolutions
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Context Encoding for Semantic Segmentation
• Learning a Discriminative Feature Network for Semantic Segmentation
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation
• Learning Markov Clustering Networks for Scene Text Detection
• LabelBank: Revisiting Global Perspectives for Semantic Segmentation
• Scene Parsing with Global Context Embedding
• Feature Boosting Network For 3D Pose Estimation
• Error Correction for Dense Semantic Image Labeling
• A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation
• A Classification Refinement Strategy for Semantic Segmentation

Cited by in corpus (3)

• Polarized Self-Attention: Towards High-quality Pixel-wise Regression
• GFF: Gated Fully Fusion for Semantic Segmentation
• Multi-Scale Feature Aggregation by Cross-Scale Pixel-to-Region Relation Operation for Semantic Segmentation