Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge

Abstract

Automatic detection of pulmonary nodules in thoracic computed tomography (CT) scans has been an active area of research for the last two decades. However, there have only been few studies that provide a comparative performance evaluation of different systems on a common database. We have therefore set up the LUNA16 challenge, an objective evaluation framework for automatic nodule detection algorithms using the largest publicly available reference database of chest CT scans, the LIDC-IDRI data set. In LUNA16, participants develop their algorithm and upload their predictions on 888 CT scans in one of the two tracks: 1) the complete nodule detection track where a complete CAD system should be developed, or 2) the false positive reduction track where a provided set of nodule candidates should be classified. This paper describes the setup of LUNA16 and presents the results of the challenge so far. Moreover, the impact of combining individual systems on the detection performance was also investigated. It was observed that the leading solutions employed convolutional networks and used the provided set of nodule candidates. The combination of these solutions achieved an excellent sensitivity of over 95% at fewer than 1.0 false positives per scan. This highlights the potential of combining algorithms to improve the detection performance. Our observer study with four expert readers has shown that the best system detects nodules that were missed by expert readers who originally annotated the LIDC-IDRI data. We released this set of additional nodules for further development of CAD systems.

References in corpus (6)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Improving neural networks by preventing co-adaptation of feature detectors
• Wide Residual Networks
• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• Theano: new features and speed improvements
• The Ladder: A Reliable Leaderboard for Machine Learning Competitions

Cited by in corpus (97)

• MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation
• Recent advances and clinical applications of deep learning in medical image analysis
• fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
• Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network
• Segment Anything Model for Medical Images?
• Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations
• Hybrid Deep Learning for Detecting Lung Diseases from X-ray Images
• Why rankings of biomedical image analysis competitions should be interpreted with care
• Models Genesis
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Lung and Pancreatic Tumor Characterization in the Deep Learning Era: Novel Supervised and Unsupervised Learning Approaches
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• Automatic Pulmonary Nodule Detection in CT Scans Using Convolutional Neural Networks Based on Maximum Intensity Projection
• Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules
• nnDetection: A Self-configuring Method for Medical Object Detection
• Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays
• Reinventing 2D Convolutions for 3D Images
• SCPM-Net: An Anchor-free 3D Lung Nodule Detection Network using Sphere Representation and Center Points Matching
• A Deep Learning System That Generates Quantitative CT Reports for Diagnosing Pulmonary Tuberculosis
• Integration of Convolutional Neural Networks for Pulmonary Nodule Malignancy Assessment in a Lung Cancer Classification Pipeline
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• 3D G-CNNs for Pulmonary Nodule Detection
• HybridMIM: A Hybrid Masked Image Modeling Framework for 3D Medical Image Segmentation
• Deep Learning for Lung Cancer Detection: Tackling the Kaggle Data Science Bowl 2017 Challenge
• DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification
• Evolving the pulmonary nodules diagnosis from classical approaches to deep learning aided decision support: three decades development course and future prospect
• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks
• Deep convolutional neural networks for multi-planar lung nodule detection: improvement in small nodule identification
• Unsupervised Contrastive Learning based Transformer for Lung Nodule Detection
• FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
• Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization
• Lung Cancer Diagnosis Using Deep Attention Based on Multiple Instance Learning and Radiomics
• Interpretative Computer-aided Lung Cancer Diagnosis: from Radiology Analysis to Malignancy Evaluation
• Propagating Uncertainty in Multi-Stage Bayesian Convolutional Neural Networks with Application to Pulmonary Nodule Detection
• CASED: Curriculum Adaptive Sampling for Extreme Data Imbalance
• Relational Learning between Multiple Pulmonary Nodules via Deep Set Attention Transformers
• Improving Representation of High-frequency Components for Medical Visual Foundation Models
• Decompose to manipulate: Manipulable Object Synthesis in 3D Medical Images with Structured Image Decomposition
• Evaluating the Robustness of Self-Supervised Learning in Medical Imaging
• Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
• 2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data
• FocalMix: Semi-Supervised Learning for 3D Medical Image Detection
• Coronary Artery Segmentation from Intravascular Optical Coherence Tomography Using Deep Capsules
• Deep learning-based conditional inpainting for restoration of artifact-affected 4D CT images
• U-Det: A Modified U-Net architecture with bidirectional feature network for lung nodule segmentation
• AutoLungDx: A Hybrid Deep Learning Approach for Early Lung Cancer Diagnosis Using 3D Res-U-Net, YOLOv5, and Vision Transformers
• Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning
• Dual-branch residual network for lung nodule segmentation
• Virtual Lung Screening Trial (VLST): An In Silico Study Inspired by the National Lung Screening Trial for Lung Cancer Detection
• DeepEM: Deep 3D ConvNets With EM For Weakly Supervised Pulmonary Nodule Detection
• Validated respiratory drug deposition predictions from 2D and 3D medical images with statistical shape models and convolutional neural networks
• Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT
• A Systematic Collection of Medical Image Datasets for Deep Learning
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation
• NoduleNet: Decoupled False Positive Reductionfor Pulmonary Nodule Detection and Segmentation
• Two-Stage Convolutional Neural Network Architecture for Lung Nodule Detection
• Needles in Haystacks: On Classifying Tiny Objects in Large Images
• Improving Breast Cancer Detection using Symmetry Information with Deep Learning
• Universal Lesion Detection by Learning from Multiple Heterogeneously Labeled Datasets
• Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration
• RMSim: Controlled Respiratory Motion Simulation on Static Patient Scans
• No Surprises: Training Robust Lung Nodule Detection for Low-Dose CT Scans by Augmenting with Adversarial Attacks
• DeepLung: 3D Deep Convolutional Nets for Automated Pulmonary Nodule Detection and Classification
• Towards Single-phase Single-stage Detection of Pulmonary Nodules in Chest CT Imaging
• Integration of Federated Learning and Blockchain in Healthcare: A Tutorial
• A new semi-supervised self-training method for lung cancer prediction
• False Positive Reduction in Lung Computed Tomography Images using Convolutional Neural Networks
• Automatic Pulmonary Lobe Segmentation Using Deep Learning
• A Pulmonary Nodule Detection Model Based on Progressive Resolution and Hierarchical Saliency
• DeepSEED: 3D Squeeze-and-Excitation Encoder-Decoder Convolutional Neural Networks for Pulmonary Nodule Detection
• S4ND: Single-Shot Single-Scale Lung Nodule Detection
• An Efficient Approach to Detecting Lung Nodules Using Swin Transformer
• MeDaS: An open-source platform as service to help break the walls between medicine and informatics
• Deep Learning for Automated Medical Image Analysis
• Random Bundle: Brain Metastases Segmentation Ensembling through Annotation Randomization
• 3DFPN-HS${}^2$: 3D Feature Pyramid Network Based High Sensitivity and Specificity Pulmonary Nodule Detection
• Survey of the Detection and Classification of Pulmonary Lesions via CT and X-Ray
• Medical Datasets Collections for Artificial Intelligence-based Medical Image Analysis
• Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• The Missing Piece: A Case for Pre-Training in 3D Medical Object Detection
• A Deep Regression Model for Seed Identification in Prostate Brachytherapy
• PiaNet: A pyramid input augmented convolutional neural network for GGO detection in 3D lung CT scans
• Integrating Feature and Image Pyramid: A Lung Nodule Detector Learned in Curriculum Fashion
• Did you miss it? Automatic lung nodule detection combined with gaze information improves radiologists' screening performance
• Multi-Scale Gradual Integration CNN for False Positive Reduction in Pulmonary Nodule Detection
• Primary Tumor Origin Classification of Lung Nodules in Spectral CT using Transfer Learning
• Pathology-Aware Generative Adversarial Networks for Medical Image Augmentation
• Pulmonary Nodule Malignancy Classification Using its Temporal Evolution with Two-Stream 3D Convolutional Neural Networks
• An End-to-end Framework For Integrated Pulmonary Nodule Detection and False Positive Reduction
• Automated pulmonary nodule detection using 3D deep convolutional neural networks
• Multi-task Semi-supervised Learning for Pulmonary Lobe Segmentation
• Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontology
• An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth
• Exploring Instance-Level Uncertainty for Medical Detection
• Deep Generative Classifiers for Thoracic Disease Diagnosis with Chest X-ray Images