Prototype Refinement Network for Few-Shot Segmentation

Abstract

Few-shot segmentation targets to segment new classes with few annotated images provided. It is more challenging than traditional semantic segmentation tasks that segment known classes with abundant annotated images. In this paper, we propose a Prototype Refinement Network (PRNet) to attack the challenge of few-shot segmentation. It firstly learns to bidirectionally extract prototypes from both support and query images of the known classes. Furthermore, to extract representative prototypes of the new classes, we use adaptation and fusion for prototype refinement. The step of adaptation makes the model to learn new concepts which is directly implemented by retraining. Prototype fusion is firstly proposed which fuses support prototypes with query prototypes, incorporating the knowledge from both sides. It is effective in prototype refinement without importing extra learnable parameters. In this way, the prototypes become more discriminative in low-data regimes. Experiments on PASAL-$5^i$ and COCO-$20^i$ demonstrate the superiority of our method. Especially on COCO-$20^i$, PRNet significantly outperforms existing methods by a large margin of 13.1\% in 1-shot setting.

References in corpus (6)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment
• SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation
• Infinite Mixture Prototypes for Few-Shot Learning

Cited by in corpus (5)

• FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network
• Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation
• A Location-Sensitive Local Prototype Network for Few-Shot Medical Image Segmentation
• Few-Shot Semantic Segmentation Augmented with Image-Level Weak Annotations
• Prototypical Region Proposal Networks for Few-Shot Localization and Classification