Consensus Feature Network for Scene Parsing

Abstract

Scene parsing is challenging as it aims to assign one of the semantic categories to each pixel in scene images. Thus, pixel-level features are desired for scene parsing. However, classification networks are dominated by the discriminative portion, so directly applying classification networks to scene parsing will result in inconsistent parsing predictions within one instance and among instances of the same category. To address this problem, we propose two transform units to learn pixel-level consensus features. One is an Instance Consensus Transform (ICT) unit to learn the instance-level consensus features by aggregating features within the same instance. The other is a Category Consensus Transform (CCT) unit to pursue category-level consensus features through keeping the consensus of features among instances of the same category in scene images. The proposed ICT and CCT units are lightweight, data-driven and end-to-end trainable. The features learned by the two units are more coherent in both instance-level and category-level. Furthermore, we present the Consensus Feature Network (CFNet) based on the proposed ICT and CCT units, and demonstrate the effectiveness of each component in our method by performing extensive ablation experiments. Finally, our proposed CFNet achieves competitive performance on four datasets, including Cityscapes, Pascal Context, CamVid, and COCO Stuff.

arXiv comment13 pages, 7 figures, 9 tables

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rethinking Atrous Convolution for Semantic Image Segmentation
• ParseNet: Looking Wider to See Better
• OCNet: Object Context Network for Scene Parsing
• ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
• Dual Attention Network for Scene Segmentation
• CGNet: A Light-weight Context Guided Network for Semantic Segmentation
• SPGNet: Semantic Prediction Guidance for Scene Parsing
• Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network
• Learning Dense Convolutional Embeddings for Semantic Segmentation
• FoveaNet: Perspective-aware Urban Scene Parsing
• Tree-structured Kronecker Convolutional Network for Semantic Segmentation

Cited by in corpus (1)

• GINet: Graph Interaction Network for Scene Parsing