Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training

Abstract

To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire because labeled medical images are not usually available due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions and poor standardization. Lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose an alternative framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and hypothesize that clinically-relevant features can be preserved via self-regularization. These domain-adapted images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We test this approach for the notoriously difficult task of depth-estimation from endoscopy. We train a depth estimator on a large dataset of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. This network predicts significantly better depths when using synthetic-like domain-adapted images compared to the real images, confirming that the clinically-relevant features of depth are preserved.

10 pages, 8 figure

References in corpus (4)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• Learning from Synthetic Humans
• Depth Reconstruction and Computer-Aided Polyp Detection in Optical Colonoscopy Video Frames

Cited by in corpus (20)

• Generative Adversarial Network in Medical Imaging: A Review
• Domain Adaptation for Medical Image Analysis: A Survey
• Boundary-weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation
• Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy
• CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions
• Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classification
• Abnormal Colon Polyp Image Synthesis Using Conditional Adversarial Networks for Improved Detection Performance
• Deep learning in biomedical optics
• Automatic Segmentation and Visualization of Choroid in OCT with Knowledge Infused Deep Learning
• Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation
• Bimodal Camera Pose Prediction for Endoscopy
• DeepLSR: a deep learning approach for laser speckle reduction
• Scribble-based Domain Adaptation via Co-segmentation
• Style-transfer GANs for bridging the domain gap in synthetic pose estimator training
• LightDepth: Single-View Depth Self-Supervision from Illumination Decline
• Simple U-net Based Synthetic Polyp Image Generation: Polyp to Negative and Negative to Polyp
• GAN Inversion for Data Augmentation to Improve Colonoscopy Lesion Classification
• Semi-supervised classification of radiology images with NoTeacher: A Teacher that is not Mean
• This Intestine Does Not Exist: Multiscale Residual Variational Autoencoder for Realistic Wireless Capsule Endoscopy Image Generation
• Multiscale Vector-Quantized Variational Autoencoder for Endoscopic Image Synthesis