Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation

Abstract

The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations. In this article, we provide a detailed review of the solutions above, summarizing both the technical novelties and empirical results. We further compare the benefits and requirements of the surveyed methodologies and provide our recommended solutions. We hope this survey article increases the community awareness of the techniques that are available to handle imperfect medical image segmentation datasets.

Accepted for publication in the journal of Medical Image Analysis

References in corpus (35)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Understanding deep learning requires rethinking generalization
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy
• Patch-based Output Space Adversarial Learning for Joint Optic Disc and Cup Segmentation
• Fully Connected Deep Structured Networks
• Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations
• DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
• Synthetic Medical Images from Dual Generative Adversarial Networks
• XLSor: A Robust and Accurate Lung Segmentor on Chest X-Rays Using Criss-Cross Attention and Customized Radiorealistic Abnormalities Generation
• Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation
• Cost-Effective Active Learning for Melanoma Segmentation
• Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation
• Interactive segmentation of medical images through fully convolutional neural networks
• Data augmentation using learned transformations for one-shot medical image segmentation
• PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network with a Benchmark at Cross-modality Cardiac Segmentation
• Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis
• Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation
• When Unseen Domain Generalization is Unnecessary? Rethinking Data Augmentation
• Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation
• A Survey on Deep Learning of Small Sample in Biomedical Image Analysis
• Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction
• Boundary and Entropy-driven Adversarial Learning for Fundus Image Segmentation
• A sparse annotation strategy based on attention-guided active learning for 3D medical image segmentation
• Self-Supervised Learning for Spinal MRIs
• Multi-Task Attention-Based Semi-Supervised Learning for Medical Image Segmentation
• Surrogate Supervision for Medical Image Analysis: Effective Deep Learning From Limited Quantities of Labeled Data
• Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST
• Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model
• Transfer Learning with Edge Attention for Prostate MRI Segmentation
• Recurrent Aggregation Learning for Multi-View Echocardiographic Sequences Segmentation
• Neural Style Transfer Improves 3D Cardiovascular MR Image Segmentation on Inconsistent Data
• Pick-and-Learn: Automatic Quality Evaluation for Noisy-Labeled Image Segmentation
• Self-Learning to Detect and Segment Cysts in Lung CT Images without Manual Annotation

Cited by in corpus (60)

• Recent advances and clinical applications of deep learning in medical image analysis
• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis
• Annotation-efficient deep learning for automatic medical image segmentation
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions
• BS-Net: learning COVID-19 pneumonia severity on a large Chest X-Ray dataset
• Reinventing 2D Convolutions for 3D Images
• Review: Deep Learning in Electron Microscopy
• Fully Automated Multi-Organ Segmentation in Abdominal Magnetic Resonance Imaging with Deep Neural Networks
• Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention
• UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
• A Unified Framework for Generalized Low-Shot Medical Image Segmentation with Scarce Data
• UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation
• medigan: a Python library of pretrained generative models for medical image synthesis
• MT-UDA: Towards Unsupervised Cross-modality Medical Image Segmentation with Limited Source Labels
• PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation
• Hierarchical Consistency Regularized Mean Teacher for Semi-supervised 3D Left Atrium Segmentation
• Constrained domain adaptation for Image segmentation
• CarveMix: A Simple Data Augmentation Method for Brain Lesion Segmentation
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks
• Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training
• Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images
• Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps
• FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
• Terabyte-scale supervised 3D training and benchmarking dataset of the mouse kidney
• Emergency Department Decision Support using Clinical Pseudo-notes
• Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT
• Dual-Task Mutual Learning for Semi-Supervised Medical Image Segmentation
• Self-Supervised Learning for 3D Medical Image Analysis using 3D SimCLR and Monte Carlo Dropout
• Universal Lesion Detection by Learning from Multiple Heterogeneously Labeled Datasets
• AGMB-Transformer: Anatomy-Guided Multi-Branch Transformer Network for Automated Evaluation of Root Canal Therapy
• Deep Mining External Imperfect Data for Chest X-ray Disease Screening
• Extreme Consistency: Overcoming Annotation Scarcity and Domain Shifts
• Critical Assessment of Transfer Learning for Medical Image Segmentation with Fully Convolutional Neural Networks
• Exploiting Shared Knowledge from Non-COVID Lesions for Annotation-Efficient COVID-19 CT Lung Infection Segmentation
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• A novel open-source ultrasound dataset with deep learning benchmarks for spinal cord injury localization and anatomical segmentation
• Contour Transformer Network for One-shot Segmentation of Anatomical Structures
• A Location-Sensitive Local Prototype Network for Few-Shot Medical Image Segmentation
• TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors
• Tips and Tricks to Improve CNN-based Chest X-ray Diagnosis: A Survey
• Look at here : Utilizing supervision to attend subtle key regions
• Aggregative Self-Supervised Feature Learning from a Limited Sample
• Deep Structure Learning using Feature Extraction in Trained Projection Space
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Generalizing electrocardiogram delineation -- Training convolutional neural networks with synthetic data augmentation
• Deep Learning for Earth Image Segmentation based on Imperfect Polyline Labels with Annotation Errors
• Classifying Breast Histopathology Images with a Ductal Instance-Oriented Pipeline
• Hetero-Modal Learning and Expansive Consistency Constraints for Semi-Supervised Detection from Multi-Sequence Data
• A Research Agenda on Pediatric Chest X-Ray: Is Deep Learning Still in Childhood?
• Dual Shape Guided Segmentation Network for Organs-at-Risk in Head and Neck CT Images
• The Power of Proxy Data and Proxy Networks for Hyper-Parameter Optimization in Medical Image Segmentation
• ErrorNet: Learning error representations from limited data to improve vascular segmentation
• One-shot Weakly-Supervised Segmentation in Medical Images
• Vessel-CAPTCHA: an efficient learning framework for vessel annotation and segmentation
• Unpaired Single-Image Depth Synthesis with cycle-consistent Wasserstein GANs
• OXnet: Omni-supervised Thoracic Disease Detection from Chest X-rays
• Automated Multi-Channel Segmentation for the 4D Myocardial Velocity Mapping Cardiac MR