Segment Anything in Medical Images
arXiv:2304.12306 · doi:10.1038/s41467-024-44824-z
Abstract
Medical image segmentation is a critical component in clinical practice, facilitating accurate diagnosis, treatment planning, and disease monitoring. However, existing methods, often tailored to specific modalities or disease types, lack generalizability across the diverse spectrum of medical image segmentation tasks. Here we present MedSAM, a foundation model designed for bridging this gap by enabling universal medical image segmentation. The model is developed on a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. We conduct a comprehensive evaluation on 86 internal validation tasks and 60 external validation tasks, demonstrating better accuracy and robustness than modality-wise specialist models. By delivering accurate and efficient segmentation across a wide spectrum of tasks, MedSAM holds significant potential to expedite the evolution of diagnostic tools and the personalization of treatment plans.
References in corpus (12)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Automated Design of Deep Learning Methods for Biomedical Image Segmentation
- The Medical Segmentation Decathlon
- Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
- Segment Anything Model for Medical Image Analysis: an Experimental Study
- DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
- Segment Anything Model for Medical Images?
- Metrics reloaded: Recommendations for image analysis validation
- MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning
- SAM Struggles in Concealed Scenes -- Empirical Study on Segment Anything
- Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging
- Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets
Cited by in corpus (84)
- Segment Anything Model for Medical Image Analysis: an Experimental Study
- Segment Anything Model for Medical Images?
- The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions
- Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey
- 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
- BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once
- TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI
- Foundation Models in Radiology: What, How, When, Why and Why Not
- A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation
- A large-scale multicenter breast cancer DCE-MRI benchmark dataset with expert segmentations
- Artificial General Intelligence for Medical Imaging Analysis
- Deep learning based infrared small object segmentation: Challenges and future directions
- Unleashing the Power of Generic Segmentation Models: A Simple Baseline for Infrared Small Target Detection
- Pixel super-resolved virtual staining of label-free tissue using diffusion models
- Evaluating Pre-trained Convolutional Neural Networks and Foundation Models as Feature Extractors for Content-based Medical Image Retrieval
- Embedded Visual Prompt Tuning
- Deep Learning for Pancreas Segmentation: a Systematic Review
- Improving Representation of High-frequency Components for Medical Visual Foundation Models
- Data-Centric Foundation Models in Computational Healthcare: A Survey
- MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis
- PaveSAM Segment Anything for Pavement Distress
- HSMix: Hard and Soft Mixing Data Augmentation for Medical Image Segmentation
- Is SAM 2 Better than SAM in Medical Image Segmentation?
- UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets
- Sam2Rad: A Segmentation Model for Medical Images with Learnable Prompts
- PECI-Net: Bolus segmentation from video fluoroscopic swallowing study images using preprocessing ensemble and cascaded inference
- MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models
- Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)
- Scaling up self-supervised learning for improved surgical foundation models
- Local Differential Privacy is Not Enough: A Sample Reconstruction Attack against Federated Learning with Local Differential Privacy
- NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation
- Discriminative Hamiltonian Variational Autoencoder for Accurate Tumor Segmentation in Data-Scarce Regimes
- Multitask Learning in Minimally Invasive Surgical Vision: A Review
- Generalizing Medical Image Representations via Quaternion Wavelet Networks
- SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance
- Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Models
- Zero-shot 3D Segmentation of Abdominal Organs in CT Scans Using Segment Anything Model 2: Adapting Video Tracking Capabilities for 3D Medical Imaging
- Evaluation of state-of-the-art deep learning models in the segmentation of the heart ventricles in parasternal short-axis echocardiograms
- How to Efficiently Annotate Images for Best-Performing Deep Learning Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Model
- DuetMatch: Harmonizing Semi-Supervised Brain MRI Segmentation via Decoupled Branch Optimization
- Extreme Cardiac MRI Analysis under Respiratory Motion: Results of the CMRxMotion Challenge
- Frontiers in Intelligent Colonoscopy
- Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
- Adapting SAM for Volumetric X-Ray Data-sets of Arbitrary Sizes
- ONCOPILOT: A Promptable CT Foundation Model For Solid Tumor Evaluation
- Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging
- DualStreamFoveaNet: A Dual Stream Fusion Architecture with Anatomical Awareness for Robust Fovea Localization
- The Explanation Necessity for Healthcare AI
- SAMRI-2: A Memory-based Model for Cartilage and Meniscus Segmentation in 3D MRIs of the Knee Joint
- SASVi -- Segment Any Surgical Video
- 3D Foundation Model for Generalizable Disease Detection in Head Computed Tomography
- Cinepro: Robust Training of Foundation Models for Cancer Detection in Prostate Ultrasound Cineloops
- Adaptive Mix for Semi-Supervised Medical Image Segmentation
- Generative AI and Foundation Models in Medical Image
- MRAnnotator: multi-Anatomy and many-Sequence MRI segmentation of 44 structures
- Muographic Image Upsampling with Machine Learning for Built Infrastructure Applications
- FACT: Foundation Model for Assessing Cancer Tissue Margins with Mass Spectrometry
- Sparse Mixture-of-Experts for Non-Uniform Noise Reduction in MRI Images
- MRUCT: Mixed Reality Assistance for Acupuncture Guided by Ultrasonic Computed Tomography
- HICEScore: A Hierarchical Metric for Image Captioning Evaluation
- Performance of the Segment Anything Model in Various RFI/Events Detection in Radio Astronomy
- An Active Learning Pipeline for Biomedical Image Instance Segmentation with Minimal Human Intervention
- Utilizing Grounded SAM for self-supervised frugal camouflaged human detection
- FA-Seg: A Fast and Accurate Diffusion-Based Method for Open-Vocabulary Segmentation
- VIS-MAE: An Efficient Self-supervised Learning Approach on Medical Image Segmentation and Classification
- Safety-Ensured Robotic Control Framework for Cutting Task Automation in Endoscopic Submucosal Dissection
- Med-PU: Point Cloud Upsampling for High-Fidelity 3D Medical Shape Reconstruction
- Advancing Positron Emission Tomography Image Quantification: Artificial Intelligence-Driven Methods, Clinical Challenges, and Emerging Opportunities in Long-Axial Field-of-View Positron Emission Tomography/Computed Tomography Imaging
- SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation
- Quantifying Knee Cartilage Shape and Lesion: From Image to Metrics
- Unified Medical Image Segmentation with State Space Modeling Snake
- Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages
- MedSymmFlow: Bridging Generative Modeling and Classification in Medical Imaging through Symmetrical Flow Matching
- FLIM Networks with Bag of Feature Points
- Training-free Temporal Object Tracking in Surgical Videos
- DyABD: The Abdominal Muscle Segmentation in Dynamic MRI Benchmark
- Enhancing Weakly Supervised Semantic Segmentation for Fibrosis via Controllable Image Generation
- Innovative Quantitative Analysis for Disease Progression Assessment in Familial Cerebral Cavernous Malformations
- DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studies
- A Comparison of Deep Learning Methods for Cell Detection in Digital Cytology
- AMLP: Adjustable Masking Lesion Patches for Self-Supervised Medical Image Segmentation
- The TYC Dataset for Understanding Instance-Level Semantics and Motions of Cells in Microstructures
- Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation
- BreastSegNet: Multi-label Segmentation of Breast MRI