Fully Convolutional Networks for Panoptic Segmentation

Abstract

In this paper, we present a conceptually simple, strong, and efficient framework for panoptic segmentation, called Panoptic FCN. Our approach aims to represent and predict foreground things and background stuff in a unified fully convolutional pipeline. In particular, Panoptic FCN encodes each object instance or stuff category into a specific kernel weight with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly. With this approach, instance-aware and semantically consistent properties for things and stuff can be respectively satisfied in a simple generate-kernel-then-segment workflow. Without extra boxes for localization or instance separation, the proposed approach outperforms previous box-based and -free models with high efficiency on COCO, Cityscapes, and Mapillary Vistas datasets with single scale input. Our code is made publicly available at https://github.com/Jia-Research-Lab/PanopticFCN.

arXiv commentCVPR2021 Oral

References in corpus (15)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Path Aggregation Network for Instance Segmentation
• DeeperLab: Single-Shot Image Parser
• Learning to Fuse Things and Stuff
• Panoptic Feature Pyramid Networks
• SSAP: Single-Shot Instance Segmentation With Affinity Pyramid
• Conditional Convolutions for Instance Segmentation
• Learnable Tree Filter for Structure-preserving Feature Transform
• Learning Dynamic Routing for Semantic Segmentation
• BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation
• Bidirectional Graph Reasoning Network for Panoptic Segmentation
• PointINS: Point-based Instance Segmentation