Segment Anything

Abstract

We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date (by far), with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive -- often competitive with or even superior to prior fully supervised results. We are releasing the Segment Anything Model (SAM) and corresponding dataset (SA-1B) of 1B masks and 11M images at https://segment-anything.com to foster research into foundation models for computer vision.

arXiv commentProject web-page: https://segment-anything.com

Cited by in corpus (32)

• Materials science in the era of large language models: a perspective
• Vision Transformers Need Registers
• FoodSAM: Any Food Segmentation
• Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing
• Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models
• Keypoint Promptable Re-Identification
• ExpressEdit: Video Editing with Natural Language and Sketching
• Vehicle Perception from Satellite
• Label-free segmentation from cardiac ultrasound using self-supervised learning
• Automated Grain Boundary Detection for Bright-Field Transmission Electron Microscopy Images via U-Net
• Structure in Deep Reinforcement Learning: A Survey and Open Problems
• Automated Measurement of Vascular Calcification in Femoral Endarterectomy Patients Using Deep Learning
• OMG-Net: A Deep Learning Framework Deploying Segment Anything to Detect Pan-Cancer Mitotic Figures from Haematoxylin and Eosin-Stained Slides
• MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback
• Physics-informed generative model for drug-like molecule conformers
• Automatic view plane prescription for cardiac magnetic resonance imaging via supervision by spatial relationship between views
• Relational Programming with Foundation Models
• SAMBA: A Trainable Segmentation Web-App with Smart Labelling
• MetaFBP: Learning to Learn High-Order Predictor for Personalized Facial Beauty Prediction
• Processing and Segmentation of Human Teeth from 2D Images using Weakly Supervised Learning
• Find Rhinos without Finding Rhinos: Active Learning with Multimodal Imagery of South African Rhino Habitats
• Leveraging Self-Supervised Vision Transformers for Segmentation-based Transfer Function Design
• Pattern Guided UV Recovery for Realistic Video Garment Texturing
• POPCat: Propagation of particles for complex annotation tasks
• Space-scale Exploration of the Poor Reliability of Deep Learning Models: the Case of the Remote Sensing of Rooftop Photovoltaic Systems
• A SAM based Tool for Semi-Automatic Food Annotation
• The Teenager's Problem: Efficient Garment Decluttering as Probabilistic Set Cover
• Interpreting Equivariant Representations
• Fluorescence Molecular Tomography for Quantum Yield and Lifetime
• Semantic Generative Augmentations for Few-Shot Counting
• Zero-shot Segmentation of Skin Conditions: Erythema with Edit-Friendly Inversion
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation