Error Correction for Dense Semantic Image Labeling

Abstract

Pixelwise semantic image labeling is an important, yet challenging, task with many applications. Typical approaches to tackle this problem involve either the training of deep networks on vast amounts of images to directly infer the labels or the use of probabilistic graphical models to jointly model the dependencies of the input (i.e. images) and output (i.e. labels). Yet, the former approaches do not capture the structure of the output labels, which is crucial for the performance of dense labeling, and the latter rely on carefully hand-designed priors that require costly parameter tuning via optimization techniques, which in turn leads to long inference times. To alleviate these restrictions, we explore how to arrive at dense semantic pixel labels given both the input image and an initial estimate of the output labels. We propose a parallel architecture that: 1) exploits the context information through a LabelPropagation network to propagate correct labels from nearby pixels to improve the object boundaries, 2) uses a LabelReplacement network to directly replace possibly erroneous, initial labels with new ones, and 3) combines the different intermediate results via a Fusion network to obtain the final per-pixel label. We experimentally validate our approach on two different datasets for the semantic segmentation and face parsing tasks respectively, where we show improvements over the state-of-the-art. We also provide both a quantitative and qualitative analysis of the generated results.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Going Deeper with Convolutions
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• ParseNet: Looking Wider to See Better
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• Pyramid Scene Parsing Network
• Attention to Scale: Scale-aware Semantic Image Segmentation
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
• Iterative Instance Segmentation

Cited by in corpus (2)

• Semantic Correlation Promoted Shape-Variant Context for Segmentation
• Learning to Refine Human Pose Estimation