UPSNet: A Unified Panoptic Segmentation Network

Abstract

In this paper, we propose a unified panoptic segmentation network (UPSNet) for tackling the newly proposed panoptic segmentation task. On top of a single backbone residual network, we first design a deformable convolution based semantic segmentation head and a Mask R-CNN style instance segmentation head which solve these two subtasks simultaneously. More importantly, we introduce a parameter-free panoptic head which solves the panoptic segmentation via pixel-wise classification. It first leverages the logits from the previous two heads and then innovatively expands the representation for enabling prediction of an extra unknown class which helps better resolve the conflicts between semantic and instance segmentation. Additionally, it handles the challenge caused by the varying number of instances and permits back propagation to the bottom modules in an end-to-end manner. Extensive experimental results on Cityscapes, COCO and our internal dataset demonstrate that our UPSNet achieves state-of-the-art performance with much faster inference. Code has been made available at: https://github.com/uber-research/UPSNet

CVPR 2019

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• Horovod: fast and easy distributed deep learning in TensorFlow
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Learning to Fuse Things and Stuff
• Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
• MegDet: A Large Mini-Batch Object Detector
• Attention-guided Unified Network for Panoptic Segmentation

Cited by in corpus (9)

• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities
• Attention-guided Unified Network for Panoptic Segmentation
• SpatialFlow: Bridging All Tasks for Panoptic Segmentation
• Bidirectional Graph Reasoning Network for Panoptic Segmentation
• IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things
• Generator evaluator-selector net for panoptic image segmentation and splitting unfamiliar objects into parts
• Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN