Monocular Object Instance Segmentation and Depth Ordering with CNNs

Abstract

In this paper we tackle the problem of instance-level segmentation and depth ordering from a single monocular image. Towards this goal, we take advantage of convolutional neural nets and train them to directly predict instance-level segmentations where the instance ID encodes the depth ordering within image patches. To provide a coherent single explanation of an image we develop a Markov random field which takes as input the predictions of convolutional neural nets applied at overlapping patches of different resolutions, as well as the output of a connected component algorithm. It aims to predict accurate instance-level segmentation and depth ordering. We demonstrate the effectiveness of our approach on the challenging KITTI benchmark and show good performance on both tasks.

International Conference on Computer Vision (ICCV), 2015

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Fully Convolutional Networks for Semantic Segmentation
• DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Fully Connected Deep Structured Networks
• Deep Visual-Semantic Alignments for Generating Image Descriptions

Cited by in corpus (26)

• Semantic Instance Segmentation with a Discriminative Loss Function
• Single-Image Depth Perception in the Wild
• Annotating Object Instances with a Polygon-RNN
• Weakly Supervised Instance Segmentation using Class Peak Response
• Deep Watershed Transform for Instance Segmentation
• Deep Active Contours
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
• InstanceCut: from Edges to Instances with MultiCut
• Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
• Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs
• Evolutionary Synthesis of Deep Neural Networks via Synaptic Cluster-driven Genetic Encoding
• Joint Graph Decomposition and Node Labeling: Problem, Algorithms, Applications
• Reversible Recursive Instance-level Object Segmentation
• FoveaNet: Perspective-aware Urban Scene Parsing
• Boundary-aware Instance Segmentation
• Semi-convolutional Operators for Instance Segmentation
• A Foreground Inference Network for Video Surveillance Using Multi-View Receptive Field
• DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
• Efficient Column Generation for Cell Detection and Segmentation
• Pixelwise Instance Segmentation with a Dynamically Instantiated Network
• SIGNet: Semantic Instance Aided Unsupervised 3D Geometry Perception
• Weakly- and Semi-Supervised Panoptic Segmentation
• CASNet: Common Attribute Support Network for image instance and panoptic segmentation
• Distance to Center of Mass Encoding for Instance Segmentation
• DDNet: Dual-path Decoder Network for Occlusion Relationship Reasoning
• Relative Depth Order Estimation Using Multi-scale Densely Connected Convolutional Networks