Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling

Abstract

Recent approaches for instance-aware semantic labeling have augmented convolutional neural networks (CNNs) with complex multi-task architectures or computationally expensive graphical models. We present a method that leverages a fully convolutional network (FCN) to predict semantic labels, depth and an instance-based encoding using each pixel's direction towards its corresponding instance center. Subsequently, we apply low-level computer vision techniques to generate state-of-the-art instance segmentation on the street scene datasets KITTI and Cityscapes. Our approach outperforms existing works by a large margin and can additionally predict absolute distances of individual instances from a monocular image as well as a pixel-level semantic labeling.

Accepted at GCPR 2016. Includes supplementary material

Cited by in corpus (26)

• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Semantic Instance Segmentation with a Discriminative Loss Function
• Semantic Instance Segmentation via Deep Metric Learning
• Simultaneous Traffic Sign Detection and Boundary Estimation using Convolutional Neural Network
• DeeperLab: Single-Shot Image Parser
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
• Annotating Object Instances with a Polygon-RNN
• Weakly Supervised Instance Segmentation using Class Peak Response
• Fully Convolutional Networks for Chip-wise Defect Detection Employing Photoluminescence Images
• Deep Watershed Transform for Instance Segmentation
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
• Instance Segmentation by Deep Coloring
• InstanceCut: from Edges to Instances with MultiCut
• Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection
• Boundary-aware Instance Segmentation
• Semi-convolutional Operators for Instance Segmentation
• Semantic Road Layout Understanding by Generative Adversarial Inpainting
• Pixelwise Instance Segmentation with a Dynamically Instantiated Network
• Distance to Center of Mass Encoding for Instance Segmentation
• Weakly- and Semi-Supervised Panoptic Segmentation
• CASNet: Common Attribute Support Network for image instance and panoptic segmentation
• Collaborative Annotation of Semantic Objects in Images with Multi-granularity Supervisions
• Bounding Box Embedding for Single Shot Person Instance Segmentation