Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation

Abstract

Although having achieved great success in medical image segmentation, deep learning-based approaches usually require large amounts of well-annotated data, which can be extremely expensive in the field of medical image analysis. Unlabeled data, on the other hand, is much easier to acquire. Semi-supervised learning and unsupervised domain adaptation both take the advantage of unlabeled data, and they are closely related to each other. In this paper, we propose uncertainty-aware multi-view co-training (UMCT), a unified framework that addresses these two tasks for volumetric medical image segmentation. Our framework is capable of efficiently utilizing unlabeled data for better performance. We firstly rotate and permute the 3D volumes into multiple views and train a 3D deep network on each view. We then apply co-training by enforcing multi-view consistency on unlabeled data, where an uncertainty estimation of each view is utilized to achieve accurate labeling. Experiments on the NIH pancreas segmentation dataset and a multi-organ segmentation dataset show state-of-the-art performance of the proposed framework on semi-supervised medical image segmentation. Under unsupervised domain adaptation settings, we validate the effectiveness of this work by adapting our multi-organ segmentation model to two pathological organs from the Medical Segmentation Decathlon Datasets. Additionally, we show that our UMCT-DA model can even effectively handle the challenging situation where labeled source data is inaccessible, demonstrating strong potentials for real-world applications.

19 pages, 6 figures, to appear in Medical Image Analysis. This article is an extension of the conference paper arXiv:1811.12506

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Explaining and Harnessing Adversarial Examples
• Rethinking Atrous Convolution for Semantic Image Segmentation
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Fully Convolutional Networks for Semantic Segmentation
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• A large annotated medical image dataset for the development and evaluation of segmentation algorithms
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Semi-Supervised Learning with Ladder Networks
• Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Learning with Pseudo-Ensembles
• nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation
• Good Semi-supervised Learning that Requires a Bad GAN
• Deep Learning with Mixed Supervision for Brain Tumor Segmentation
• Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference
• Semi-supervised Skin Lesion Segmentation via Transformation Consistent Self-ensembling Model
• Semi-Supervised Multi-Organ Segmentation via Deep Multi-Planar Co-Training
• Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation
• 3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes

Cited by in corpus (25)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Annotation-efficient deep learning for automatic medical image segmentation
• Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
• WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image
• Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation
• Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis from Lung CT Scans with Multi-Scale Guided Dense Attention
• Semi-Supervised Semantic Segmentation Based on Pseudo-Labels: A Survey
• Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentation
• Exploring Feature Representation Learning for Semi-supervised Medical Image Segmentation
• Deep Learning for Pancreas Segmentation: a Systematic Review
• MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
• External Attention Assisted Multi-Phase Splenic Vascular Injury Segmentation with Limited Data
• Semi-supervised classification of radiology images with NoTeacher: A Teacher that is not Mean
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• Progressive Dual Priori Network for Generalized Breast Tumor Segmentation
• Adaptive Hierarchical Dual Consistency for Semi-Supervised Left Atrium Segmentation on Cross-Domain Data
• Leveraging Labelled Data Knowledge: A Cooperative Rectification Learning Network for Semi-supervised 3D Medical Image Segmentation
• Generalizable Pancreas Segmentation via a Dual Self-Supervised Learning Framework
• Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT
• Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images
• SingleStrip: learning skull-stripping from a single labeled example
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation