SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

Abstract

Recently, proposal-free instance segmentation has received increasing attention due to its concise and efficient pipeline. Generally, proposal-free methods generate instance-agnostic semantic segmentation labels and instance-aware features to group pixels into different object instances. However, previous methods mostly employ separate modules for these two sub-tasks and require multiple passes for inference. We argue that treating these two sub-tasks separately is suboptimal. In fact, employing multiple separate modules significantly reduces the potential for application. The mutual benefits between the two complementary sub-tasks are also unexplored. To this end, this work proposes a single-shot proposal-free instance segmentation method that requires only one single pass for prediction. Our method is based on a pixel-pair affinity pyramid, which computes the probability that two pixels belong to the same instance in a hierarchical manner. The affinity pyramid can also be jointly learned with the semantic class labeling and achieve mutual benefits. Moreover, incorporating with the learned affinity pyramid, a novel cascaded graph partition module is presented to sequentially generate instances from coarse to fine. Unlike previous time-consuming graph partition methods, this module achieves $5\times$ speedup and 9% relative improvement on Average-Precision (AP). Our approach achieves state-of-the-art results on the challenging Cityscapes dataset.

arXiv commentICCV 2019

References in corpus (11)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Conditional Random Fields as Recurrent Neural Networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Semantic Instance Segmentation with a Discriminative Loss Function
• Semantic Instance Segmentation via Deep Metric Learning
• DeeperLab: Single-Shot Image Parser
• Proposal-free Network for Instance-level Object Segmentation

Cited by in corpus (5)

• PolyTransform: Deep Polygon Transformer for Instance Segmentation
• Fully Convolutional Networks for Panoptic Segmentation
• Real-Time Panoptic Segmentation from Dense Detections
• Boundary Knowledge Translation based Reference Semantic Segmentation
• Fast Single-shot Ship Instance Segmentation Based on Polar Template Mask in Remote Sensing Images