Exemplar-Based Open-Set Panoptic Segmentation Network
arXiv:2105.08336
Abstract
We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task. This task requires performing panoptic segmentation for not only known classes but also unknown ones that have not been acknowledged during training. We investigate the practical challenges of the task and construct a benchmark on top of an existing dataset, COCO. In addition, we propose a novel exemplar-based open-set panoptic segmentation network (EOPSN) inspired by exemplar theory. Our approach identifies a new class based on exemplars, which are identified by clustering and employed as pseudo-ground-truths. The size of each class increases by mining new exemplars based on the similarities to the existing ones associated with the class. We evaluate EOPSN on the proposed benchmark and demonstrate the effectiveness of our proposals. The primary goal of our work is to draw the attention of the community to the recognition in the open-world scenarios. The implementation of our algorithm is available on the project webpage: https://cv.snu.ac.kr/research/EOPSN.
arXiv commentCVPR 2021
References in corpus (10)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Prototypical Networks for Few-shot Learning
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- DeeperLab: Single-Shot Image Parser
- Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection
- Large-Scale Long-Tailed Recognition in an Open World
- Panoptic Feature Pyramid Networks
- An End-to-End Network for Panoptic Segmentation