Boosting Few-Shot Semantic Segmentation Via Segment Anything Model

Abstract

In semantic segmentation, accurate prediction masks are crucial for downstream tasks such as medical image analysis and image editing. Due to the lack of annotated data, few-shot semantic segmentation (FSS) performs poorly in predicting masks with precise contours. Recently, we have noticed that the large foundation model segment anything model (SAM) performs well in processing detailed features. Inspired by SAM, we propose FSS-SAM to boost FSS methods by addressing the issue of inaccurate contour. The FSS-SAM is training-free. It works as a post-processing tool for any FSS methods and can improve the accuracy of predicted masks. Specifically, we use predicted masks from FSS methods to generate prompts and then use SAM to predict new masks. To avoid predicting wrong masks with SAM, we propose a prediction result selection (PRS) algorithm. The algorithm can remarkably decrease wrong predictions. Experiment results on public datasets show that our method is superior to base FSS methods in both quantitative and qualitative aspects.

References in corpus (4)

• ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
• FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network
• Self-Regularized Prototypical Network for Few-Shot Semantic Segmentation
• Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively