Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation

Abstract

The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation due to its impressive capabilities in various segmentation tasks and its prompt-based interface. However, recent studies and individual experiments have shown that SAM underperforms in medical image segmentation, since the lack of the medical specific knowledge. This raises the question of how to enhance SAM's segmentation capability for medical images. In this paper, instead of fine-tuning the SAM model, we propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model using a light yet effective adaptation technique. In Med-SA, we propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned adaptation. We conduct comprehensive evaluation experiments on 17 medical image segmentation tasks across various image modalities. Med-SA outperforms several state-of-the-art (SOTA) medical image segmentation methods, while updating only 2\% of the parameters. Our code is released at https://github.com/KidsWithTokens/Medical-SAM-Adapter.

Code released at: https://github.com/KidsWithTokens/Medical-SAM-Adapter

Cited by in corpus (14)

• Segment Anything Model for Medical Images?
• Large AI Models in Health Informatics: Applications, Challenges, and the Future
• Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
• Deep learning based infrared small object segmentation: Challenges and future directions
• Embedded Visual Prompt Tuning
• Sam2Rad: A Segmentation Model for Medical Images with Learnable Prompts
• Achieve Fairness without Demographics for Dermatological Disease Diagnosis
• NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation
• BC-MRI-SEG: A Breast Cancer MRI Tumor Segmentation Benchmark
• Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
• DualSwinUnet++: An Enhanced Swin-Unet Architecture With Dual Decoders For PTMC Segmentation
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation