PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask

Abstract

In this work, we revisit the prior mask guidance proposed in ``Prior Guided Feature Enrichment Network for Few-Shot Segmentation''. The prior mask serves as an indicator that highlights the region of interests of unseen categories, and it is effective in achieving better performance on different frameworks of recent studies. However, the current method directly takes the maximum element-to-element correspondence between the query and support features to indicate the probability of belonging to the target class, thus the broader contextual information is seldom exploited during the prior mask generation. To address this issue, first, we propose the Context-aware Prior Mask (CAPM) that leverages additional nearby semantic cues for better locating the objects in query images. Second, since the maximum correlation value is vulnerable to noisy features, we take one step further by incorporating a lightweight Noise Suppression Module (NSM) to screen out the unnecessary responses, yielding high-quality masks for providing the prior knowledge. Both two contributions are experimentally shown to have substantial practical merit, and the new model named PFENet++ significantly outperforms the baseline PFENet as well as all other competitors on three challenging benchmarks PASCAL-5${}^i$, COCO-20${}^i$ and FSS-1000. The new state-of-the-art performance is achieved without compromising the efficiency, manifesting the potential for being a new strong baseline in few-shot semantic segmentation. Our code will be available at https://github.com/luoxiaoliu/PFENet2Plus.

The first two authors contribute equally and are listed in alphabetical order

References in corpus (14)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Prototypical Networks for Few-shot Learning
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• ParseNet: Looking Wider to See Better
• Deep High-Resolution Representation Learning for Visual Recognition
• FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
• Context Encoding for Semantic Segmentation
• Few-Shot Segmentation Propagation with Guided Networks
• SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation
• Low-Shot Learning from Imaginary Data
• CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning
• Memory Matching Networks for One-Shot Image Recognition
• Few-Shot Learning via Saliency-guided Hallucination of Samples