The Medical Segmentation Decathlon

Abstract

International challenges have become the de facto standard for comparative assessment of image analysis algorithms given a specific task. Segmentation is so far the most widely investigated medical image processing task, but the various segmentation challenges have typically been organized in isolation, such that algorithm development was driven by the need to tackle a single specific clinical problem. We hypothesized that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. To investigate the hypothesis, we organized the Medical Segmentation Decathlon (MSD) - a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities. The underlying data set was designed to explore the axis of difficulties typically encountered when dealing with medical images, such as small data sets, unbalanced labels, multi-site data and small objects. The MSD challenge confirmed that algorithms with a consistent good performance on a set of tasks preserved their good average performance on a different set of previously unseen tasks. Moreover, by monitoring the MSD winner for two years, we found that this algorithm continued generalizing well to a wide range of other clinical problems, further confirming our hypothesis. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms are mature, accurate, and generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to non AI experts.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
• Neural Architecture Search: A Survey
• AutoML: A Survey of the State-of-the-Art
• The Liver Tumor Segmentation Benchmark (LiTS)
• A large annotated medical image dataset for the development and evaluation of segmentation algorithms
• Why rankings of biomedical image analysis competitions should be interpreted with care
• Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy
• Common Limitations of Image Processing Metrics: A Picture Story
• DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentation
• Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?
• QuickNAT: A Fully Convolutional Network for Quick and Accurate Segmentation of Neuroanatomy
• AutoML Segmentation for 3D Medical Image Data: Contribution to the MSD Challenge 2018

Cited by in corpus (76)

• Segment Anything in Medical Images
• The Liver Tumor Segmentation Benchmark (LiTS)
• MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification
• Metrics reloaded: Recommendations for image analysis validation
• CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
• Recursive Cascaded Networks for Unsupervised Medical Image Registration
• H-vmunet: High-order Vision Mamba UNet for Medical Image Segmentation
• UltraLight VM-UNet: Parallel Vision Mamba Significantly Reduces Parameters for Skin Lesion Segmentation
• MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images
• Recent Progress in Transformer-based Medical Image Analysis
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
• Lumbar spine segmentation in MR images: a dataset and a public benchmark
• Robust deep learning-based semantic organ segmentation in hyperspectral images
• Deep and Statistical Learning in Biomedical Imaging: State of the Art in 3D MRI Brain Tumor Segmentation
• HybridMIM: A Hybrid Masked Image Modeling Framework for 3D Medical Image Segmentation
• Factorizer: A Scalable Interpretable Approach to Context Modeling for Medical Image Segmentation
• QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results
• Semi-Supervised Semantic Segmentation Based on Pseudo-Labels: A Survey
• Do We Really Need Dice? The Hidden Region-Size Biases of Segmentation Losses
• Implementation of a Modified U-Net for Medical Image Segmentation on Edge Devices
• Iterative pseudo-labeling based adaptive copy-paste supervision for semi-supervised tumor segmentation
• The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Cases
• MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation
• Semantic segmentation of multispectral photoacoustic images using deep learning
• Federated Multi-organ Segmentation with Inconsistent Labels
• SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images
• Brain tumour segmentation with incomplete imaging data
• DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images
• Semantic Segmentation of Anomalous Diffusion Using Deep Convolutional Networks
• Asymmetric 3D Context Fusion for Universal Lesion Detection
• RibSeg Dataset and Strong Point Cloud Baselines for Rib Segmentation from CT Scans
• Improving Representation of High-frequency Components for Medical Visual Foundation Models
• A Robust Ensemble Algorithm for Ischemic Stroke Lesion Segmentation: Generalizability and Clinical Utility Beyond the ISLES Challenge
• MambaMIM: Pre-training Mamba with State Space Token Interpolation and its Application to Medical Image Segmentation
• Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend
• HSMix: Hard and Soft Mixing Data Augmentation for Medical Image Segmentation
• 3DGAUnet: 3D generative adversarial networks with a 3D U-Net based generator to achieve the accurate and effective synthesis of clinical tumor image data for pancreatic cancer
• Towards Optimal Patch Size in Vision Transformers for Tumor Segmentation
• I${}^3$Net: Inter-Intra-slice Interpolation Network for Medical Slice Synthesis
• Phase Unwrapping of Color Doppler Echocardiography using Deep Learning
• Fed-BioMed: Open, Transparent and Trusted Federated Learning for Real-world Healthcare Applications
• Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition
• Generalizable Pancreas Segmentation via a Dual Self-Supervised Learning Framework
• Automated Segmentation of Ischemic Stroke Lesions in Non-Contrast Computed Tomography Images for Enhanced Treatment and Prognosis
• Recent Advances in Medical Image Classification
• A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation
• Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation
• BC-MRI-SEG: A Breast Cancer MRI Tumor Segmentation Benchmark
• ONCOPILOT: A Promptable CT Foundation Model For Solid Tumor Evaluation
• FESS Loss: Feature-Enhanced Spatial Segmentation Loss for Optimizing Medical Image Analysis
• VASARI-auto: equitable, efficient, and economical featurisation of glioma MRI
• Effect of the output activation function on the probabilities and errors in medical image segmentation
• View it like a radiologist: Shifted windows for deep learning augmentation of CT images
• Attention-based CT Scan Interpolation for Lesion Segmentation of Colorectal Liver Metastases
• SE3D: A Framework For Saliency Method Evaluation In 3D Imaging
• SUPER-Net: Trustworthy Image Segmentation via Uncertainty Propagation in Encoder-Decoder Networks
• Causality-inspired Single-source Domain Generalization for Medical Image Segmentation
• Heed the Noise in Performance Evaluations in Neural Architecture Search
• Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data
• Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentation
• CT respiratory motion synthesis using joint supervised and adversarial learning
• ISLE: An Intelligent Streaming Framework for High-Throughput AI Inference in Medical Imaging
• AutoML Segmentation for 3D Medical Image Data: Contribution to the MSD Challenge 2018
• Architecture Analysis and Benchmarking of 3D U-shaped Deep Learning Models for Thoracic Anatomical Segmentation
• PULASki: Learning inter-rater variability using statistical distances to improve probabilistic segmentation
• Improving Lesion Segmentation in Medical Images by Global and Regional Feature Compensation
• Advancing Positron Emission Tomography Image Quantification: Artificial Intelligence-Driven Methods, Clinical Challenges, and Emerging Opportunities in Long-Axial Field-of-View Positron Emission Tomography/Computed Tomography Imaging
• Automated Labeling of Intracranial Arteries with Uncertainty Quantification Using Deep Learning
• Multi-Resolution Feature Fusion U-Net for Magnetic Resonance Imaging Segmentation
• Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging
• Towards modelling hazard factors in unstructured data spaces using gradient-based latent interpolation
• TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiency
• Differential-UMamba: Rethinking Tumor Segmentation Under Limited Data Scenarios
• Patch-based Medical Image Segmentation using Matrix Product State Tensor Networks
• Integrating Background Knowledge in Medical Semantic Segmentation with Logic Tensor Networks
• DyABD: The Abdominal Muscle Segmentation in Dynamic MRI Benchmark