The Liver Tumor Segmentation Benchmark (LiTS)

Abstract

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in \url{http://medicaldecathlon.com/}. In addition, both data and online evaluation are accessible via \url{www.lits-challenge.com}.

arXiv commentPatrick Bilic, Patrick Christ, Hongwei Bran Li, and Eugene Vorontsov made equal contributions to this work. Published in Medical Image Analysis

References in corpus (16)

• A Survey on Deep Learning in Medical Image Analysis
• The Future of Digital Health with Federated Learning
• A "missing" family of classical orthogonal polynomials
• The Medical Segmentation Decathlon
• Fully Convolutional Networks for Semantic Segmentation
• CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• Causality matters in medical imaging
• Models Genesis
• Fully Convolutional Network Ensembles for White Matter Hyperintensities Segmentation in MR Images
• 3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes
• Disentangling Human Error from the Ground Truth in Segmentation of Medical Images
• Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects
• Conquering Data Variations in Resolution: A Slice-Aware Multi-Branch Decoder Network
• Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?
• Context-aware PolyUNet for Liver and Lesion Segmentation from Abdominal CT Images

Cited by in corpus (103)

• The Medical Segmentation Decathlon
• MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification
• Domain Adaptation for Medical Image Analysis: A Survey
• CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation
• Medical Image Segmentation Using Deep Learning: A Survey
• Segment Anything Model for Medical Image Analysis: an Experimental Study
• Segment Anything Model for Medical Images?
• MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis
• Models Genesis
• CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
• Understanding metric-related pitfalls in image analysis validation
• Reinventing 2D Convolutions for 3D Images
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
• CLIP in Medical Imaging: A Survey
• PA-ResSeg: A Phase Attention Residual Network for Liver Tumor Segmentation from Multi-phase CT Images
• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• Semi-Supervised Semantic Segmentation Based on Pseudo-Labels: A Survey
• The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Cases
• PocketNet: A Smaller Neural Network for Medical Image Analysis
• MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision
• Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation
• High Resolution Medical Image Analysis with Spatial Partitioning
• RatLesNetv2: A Fully Convolutional Network for Rodent Brain Lesion Segmentation
• Hybrid Cascaded Neural Network for Liver Lesion Segmentation
• RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR
• Conquering Data Variations in Resolution: A Slice-Aware Multi-Branch Decoder Network
• Federated brain tumor segmentation: an extensive benchmark
• A Robust Ensemble Algorithm for Ischemic Stroke Lesion Segmentation: Generalizability and Clinical Utility Beyond the ISLES Challenge
• Weakly-Supervised 3D Medical Image Segmentation using Geometric Prior and Contrastive Similarity
• Evaluating the Robustness of Self-Supervised Learning in Medical Imaging
• Hyper-Convolution Networks for Biomedical Image Segmentation
• Robust End-to-End Focal Liver Lesion Detection using Unregistered Multiphase Computed Tomography Images
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Efficient Context-Aware Network for Abdominal Multi-organ Segmentation
• Rethinking model prototyping through the MedMNIST+ dataset collection
• Liver Lesion Segmentation with slice-wise 2D Tiramisu and Tversky loss function
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
• The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism
• Harvesting, Detecting, and Characterizing Liver Lesions from Large-scale Multi-phase CT Data via Deep Dynamic Texture Learning
• Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation
• Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation
• CAFCT-Net: A CNN-Transformer Hybrid Network with Contextual and Attentional Feature Fusion for Liver Tumor Segmentation
• Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction
• Why does my medical AI look at pictures of birds? Exploring the efficacy of transfer learning across domain boundaries
• A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation
• DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets
• Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Medical Images
• Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection
• T-MPEDNet: Unveiling the Synergy of Transformer-aware Multiscale Progressive Encoder-Decoder Network with Feature Recalibration for Tumor and Liver Segmentation
• Universal Lesion Detection by Learning from Multiple Heterogeneously Labeled Datasets
• A comprehensive and easy-to-use multi-domain multi-task medical imaging meta-dataset
• Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation
• Multi-AD: Cross-Domain Unsupervised Anomaly Detection for Medical and Industrial Applications
• Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration
• How to Efficiently Annotate Images for Best-Performing Deep Learning Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Model
• E${}^2$Net: An Edge Enhanced Network for Accurate Liver and Tumor Segmentation on CT Scans
• Critical Assessment of Transfer Learning for Medical Image Segmentation with Fully Convolutional Neural Networks
• Unsupervised CT Metal Artifact Learning using Attention-guided beta-CycleGAN
• View it like a radiologist: Shifted windows for deep learning augmentation of CT images
• ONCOPILOT: A Promptable CT Foundation Model For Solid Tumor Evaluation
• MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures
• Multi-task Learning To Improve Semantic Segmentation Of CBCT Scans Using Image Reconstruction
• Joint Liver Lesion Segmentation and Classification via Transfer Learning
• Exploiting Shared Knowledge from Non-COVID Lesions for Annotation-Efficient COVID-19 CT Lung Infection Segmentation
• OnlineAugment: Online Data Augmentation with Less Domain Knowledge
• Evolution-aware VAriance (EVA) Coreset Selection for Medical Image Classification
• Context-aware PolyUNet for Liver and Lesion Segmentation from Abdominal CT Images
• Learning Shape Priors by Pairwise Comparison for Robust Semantic Segmentation
• Fully-automated deep learning slice-based muscle estimation from CT images for sarcopenia assessment
• Self-supervised Tumor Segmentation through Layer Decomposition
• Uncertainty-aware Incremental Learning for Multi-organ Segmentation
• Beyond CNNs: Exploiting Further Inherent Symmetries in Medical Images for Segmentation
• Fully-Automated Liver Tumor Localization and Characterization from Multi-Phase MR Volumes Using Key-Slice ROI Parsing: A Physician-Inspired Approach
• MeDaS: An open-source platform as service to help break the walls between medicine and informatics
• Hierarchical Self-Supervised Learning for Medical Image Segmentation Based on Multi-Domain Data Aggregation
• Implanting Synthetic Lesions for Improving Liver Lesion Segmentation in CT Exams
• KLDivNet: An unsupervised neural network for multi-modality image registration
• Task-Oriented Low-Dose CT Image Denoising
• Automatic CT Segmentation from Bounding Box Annotations using Convolutional Neural Networks
• TraceFL: Interpretability-Driven Debugging in Federated Learning via Neuron Provenance
• Generator Versus Segmentor: Pseudo-healthy Synthesis
• Anatomy-Constrained Contrastive Learning for Synthetic Segmentation without Ground-truth
• Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Brain Metastasis Segmentation Network Trained with Robustness to Annotations with Multiple False Negatives
• A New Three-stage Curriculum Learning Approach to Deep Network Based Liver Tumor Segmentation
• PET-TURTLE: Deep Unsupervised Support Vector Machines for Imbalanced Data Clusters
• A Deep Regression Model for Seed Identification in Prostate Brachytherapy
• Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
• SAR: Scale-Aware Restoration Learning for 3D Tumor Segmentation
• Rethinking Intermediate Layers design in Knowledge Distillation for Kidney and Liver Tumor Segmentation
• Spatially Dependent U-Nets: Highly Accurate Architectures for Medical Imaging Segmentation
• Towards Continuous-variable Quantum Neural Networks for Biomedical Imaging
• DyABD: The Abdominal Muscle Segmentation in Dynamic MRI Benchmark
• Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration
• Body Part Regression for CT Images
• Decoupling Shape and Density for Liver Lesion Synthesis Using Conditional Generative Adversarial Networks
• Generalized Organ Segmentation by Imitating One-shot Reasoning using Anatomical Correlation
• X-ray Dissectography Enables Stereotography to Improve Diagnostic Performance
• Segmentation overlapping wear particles with few labelled data and imbalance sample
• T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging
• Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift