Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Abstract

State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results and hardly work on unseen classes without fine-tuning. Few-shot segmentation is thus proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information of training classes and spatial inconsistency between query and support targets. To alleviate these issues, we propose the Prior Guided Feature Enrichment Network (PFENet). It consists of novel designs of (1) a training-free prior mask generation method that not only retains generalization power but also improves model performance and (2) Feature Enrichment Module (FEM) that overcomes spatial inconsistency by adaptively enriching query features with support features and prior masks. Extensive experiments on PASCAL-5${}^i$ and COCO prove that the proposed prior generation method and FEM both improve the baseline method significantly. Our PFENet also outperforms state-of-the-art methods by a large margin without efficiency loss. It is surprising that our model even generalizes to cases without labeled support samples. Our code is available at https://github.com/Jia-Research-Lab/PFENet/.

16 pages. To appear in TPAMI

References in corpus (17)

• Distributed Representations of Words and Phrases and their Compositionality
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Random Fields as Recurrent Neural Networks
• ParseNet: Looking Wider to See Better
• Deep High-Resolution Representation Learning for Visual Recognition
• FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
• Context Encoding for Semantic Segmentation
• Few-Shot Segmentation Propagation with Guided Networks
• SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation
• Low-Shot Learning from Imaginary Data
• CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning
• Memory Matching Networks for One-Shot Image Recognition
• Few-Shot Learning via Saliency-guided Hallucination of Samples
• Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates

Cited by in corpus (3)

• Anomaly Detection-Inspired Few-Shot Medical Image Segmentation Through Self-Supervision With Supervoxels
• FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network
• Iterative Few-shot Semantic Segmentation from Image Label Text