InstanceCut: from Edges to Instances with MultiCut

Abstract

This work addresses the task of instance-aware semantic segmentation. Our key motivation is to design a simple method with a new modelling-paradigm, which therefore has a different trade-off between advantages and disadvantages compared to known approaches. Our approach, we term InstanceCut, represents the problem by two output modalities: (i) an instance-agnostic semantic segmentation and (ii) all instance-boundaries. The former is computed from a standard convolutional neural network for semantic segmentation, and the latter is derived from a new instance-aware edge detection model. To reason globally about the optimal partitioning of an image into instances, we combine these two modalities into a novel MultiCut formulation. We evaluate our approach on the challenging CityScapes dataset. Despite the conceptual simplicity of our approach, we achieve the best result among all published methods, and perform particularly well for rare object classes.

The code would be released at https://github.com/alexander-kirillov/InstanceCut

References in corpus (18)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Fully Connected Deep Structured Networks
• Simultaneous Detection and Segmentation
• Semantic Image Segmentation via Deep Parsing Network
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Proposal-free Network for Instance-level Object Segmentation
• Pixel-wise Deep Learning for Contour Detection
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Instance-sensitive Fully Convolutional Networks
• Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs
• Joint Graph Decomposition and Node Labeling: Problem, Algorithms, Applications
• Reversible Recursive Instance-level Object Segmentation
• Semantic Segmentation with Boundary Neural Fields

Cited by in corpus (15)

• Semantic Instance Segmentation with a Discriminative Loss Function
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Semantic Instance Segmentation via Deep Metric Learning
• Faster Training of Mask R-CNN by Focusing on Instance Boundaries
• Fast Scene Understanding for Autonomous Driving
• Instance-level Human Parsing via Part Grouping Network
• Learning to Segment Every Thing
• Boundary-aware Instance Segmentation
• Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation
• Instance Shadow Detection
• Deep Variational Instance Segmentation
• Pixelwise Instance Segmentation with a Dynamically Instantiated Network
• Semantic Segmentation and Object Detection Towards Instance Segmentation: Breast Tumor Identification
• Bounding Box Embedding for Single Shot Person Instance Segmentation
• cvpaper.challenge in 2016: Futuristic Computer Vision through 1,600 Papers Survey