Panoptic Feature Pyramid Networks

Abstract

The recently introduced panoptic segmentation task has renewed our community's interest in unifying the tasks of instance segmentation (for thing classes) and semantic segmentation (for stuff classes). However, current state-of-the-art methods for this joint task use separate and dissimilar networks for instance and semantic segmentation, without performing any shared computation. In this work, we aim to unify these methods at the architectural level, designing a single network for both tasks. Our approach is to endow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone. Surprisingly, this simple baseline not only remains effective for instance segmentation, but also yields a lightweight, top-performing method for semantic segmentation. In this work, we perform a detailed study of this minimally extended version of Mask R-CNN with FPN, which we refer to as Panoptic FPN, and show it is a robust and accurate baseline for both tasks. Given its effectiveness and conceptual simplicity, we hope our method can serve as a strong baseline and aid future research in panoptic segmentation.

accepted to CVPR 2019

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Learning to Fuse Things and Stuff
• Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
• Triply Supervised Decoder Networks for Joint Detection and Segmentation

Cited by in corpus (32)

• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• K-Net: Towards Unified Image Segmentation
• Masked-attention Mask Transformer for Universal Image Segmentation
• MACU-Net for Semantic Segmentation of Fine-Resolution Remotely Sensed Images
• From SLAM to Situational Awareness: Challenges and Survey
• The Indirect Convolution Algorithm
• Reinforced active learning for image segmentation
• VarGNet: Variable Group Convolutional Neural Network for Efficient Embedded Computing
• Attention-guided Unified Network for Panoptic Segmentation
• Adversarial Color Enhancement: Generating Unrestricted Adversarial Images by Optimizing a Color Filter
• Fully Convolutional Networks for Panoptic Segmentation
• SpatialFlow: Bridging All Tasks for Panoptic Segmentation
• ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
• On the Integration of Self-Attention and Convolution
• Multi-Scale Feature Aggregation by Cross-Scale Pixel-to-Region Relation Operation for Semantic Segmentation
• Bidirectional Graph Reasoning Network for Panoptic Segmentation
• NETNet: Neighbor Erasing and Transferring Network for Better Single Shot Object Detection
• An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detection
• Can we cover navigational perception needs of the visually impaired by panoptic segmentation?
• SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
• Panoptic Edge Detection
• Instance and Panoptic Segmentation Using Conditional Convolutions
• Bipartite Conditional Random Fields for Panoptic Segmentation
• Exemplar-Based Open-Set Panoptic Segmentation Network
• IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things
• Detecting Reflections by Combining Semantic and Instance Segmentation
• PanoNet3D: Combining Semantic and Geometric Understanding for LiDARPoint Cloud Detection
• Generator evaluator-selector net for panoptic image segmentation and splitting unfamiliar objects into parts
• Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation
• Meta R-CNN : Towards General Solver for Instance-level Few-shot Learning
• Issues in Object Detection in Videos using Common Single-Image CNNs
• We don't Need Thousand Proposals$:$ Single Shot Actor-Action Detection in Videos