SPGNet: Semantic Prediction Guidance for Scene Parsing

Abstract

Multi-scale context module and single-stage encoder-decoder structure are commonly employed for semantic segmentation. The multi-scale context module refers to the operations to aggregate feature responses from a large spatial extent, while the single-stage encoder-decoder structure encodes the high-level semantic information in the encoder path and recovers the boundary information in the decoder path. In contrast, multi-stage encoder-decoder networks have been widely used in human pose estimation and show superior performance than their single-stage counterpart. However, few efforts have been attempted to bring this effective design to semantic segmentation. In this work, we propose a Semantic Prediction Guidance (SPG) module which learns to re-weight the local features through the guidance from pixel-wise semantic prediction. We find that by carefully re-weighting features across stages, a two-stage encoder-decoder network coupled with our proposed SPG module can significantly outperform its one-stage counterpart with similar parameters and computations. Finally, we report experimental results on the semantic segmentation benchmark Cityscapes, in which our SPGNet attains 81.1% on the test set using only 'fine' annotations.

arXiv commentICCV 2019

References in corpus (14)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Neural Architecture Search with Reinforcement Learning
• ParseNet: Looking Wider to See Better
• Fully Convolutional Networks for Semantic Segmentation
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• DeeperLab: Single-Shot Image Parser
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
• Rethinking on Multi-Stage Networks for Human Pose Estimation
• Stacked U-Nets: A No-Frills Approach to Natural Image Segmentation
• Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer

Cited by in corpus (15)

• CCNet: Criss-Cross Attention for Semantic Segmentation
• Per-Pixel Classification is Not All You Need for Semantic Segmentation
• SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
• Decoupled Classification Refinement: Hard False Positive Suppression for Object Detection
• Strip Pooling: Rethinking Spatial Pooling for Scene Parsing
• Improving Semantic Segmentation via Self-Training
• Distribution-aware Margin Calibration for Semantic Segmentation in Images
• Semantic Flow for Fast and Accurate Scene Parsing
• Inter-Image Communication for Weakly Supervised Localization
• Disentangled Non-Local Neural Networks
• Hierarchical Human Parsing with Typed Part-Relation Reasoning
• Consensus Feature Network for Scene Parsing
• Dense Dual-Path Network for Real-time Semantic Segmentation
• MPG-Net: Multi-Prediction Guided Network for Segmentation of Retinal Layers in OCT Images