An End-to-End Network for Panoptic Segmentation

Abstract

Panoptic segmentation, which needs to assign a category label to each pixel and segment each object instance simultaneously, is a challenging topic. Traditionally, the existing approaches utilize two independent models without sharing features, which makes the pipeline inefficient to implement. In addition, a heuristic method is usually employed to merge the results. However, the overlapping relationship between object instances is difficult to determine without sufficient context information during the merging process. To address the problems, we propose a novel end-to-end network for panoptic segmentation, which can efficiently and effectively predict both the instance and stuff segmentation in a single network. Moreover, we introduce a novel spatial ranking module to deal with the occlusion problem between the predicted instances. Extensive experiments have been done to validate the performance of our proposed method and promising results have been achieved on the COCO Panoptic benchmark.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Multi-Scale Context Aggregation by Dilated Convolutions
• Semantic Instance Segmentation with a Discriminative Loss Function
• Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
• Attention-guided Unified Network for Panoptic Segmentation

Cited by in corpus (8)

• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Attention-guided Unified Network for Panoptic Segmentation
• SpatialFlow: Bridging All Tasks for Panoptic Segmentation
• SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
• Exemplar-Based Open-Set Panoptic Segmentation Network
• Self-Supervised Scene De-occlusion
• Generator evaluator-selector net for panoptic image segmentation and splitting unfamiliar objects into parts
• Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation