Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging

Abstract

Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in Generative Adversarial Networks (GANs), data synthesis, and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.

arXiv commentv2, 51 pages, 15 Figures, 9 Tables, accepted for publication in Medical Image Analysis

References in corpus (72)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• A Survey on Deep Learning in Medical Image Analysis
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Denoising Diffusion Probabilistic Models
• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Bootstrap your own latent: A new approach to self-supervised Learning
• Neural Discrete Representation Learning
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Generative Adversarial Network in Medical Imaging: A Review
• Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
• NICE: Non-linear Independent Components Estimation
• Domain Generalization: A Survey
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Generative Modeling by Estimating Gradients of the Data Distribution
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• Alias-Free Generative Adversarial Networks
• A large annotated medical image dataset for the development and evaluation of segmentation algorithms
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• ResViT: Residual vision transformers for multi-modal medical image synthesis
• MedGAN: Medical Image Translation using GANs
• Semantic Segmentation using Adversarial Networks
• CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Causality matters in medical imaging
• Scalable Private Learning with PATE
• Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge
• Differentially Private Generative Adversarial Network
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• BCN20000: Dermoscopic Lesions in the Wild
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy
• Differentially Private Synthetic Medical Data Generation using Convolutional GANs
• Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
• Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
• SinGAN-Seg: Synthetic training data generation for medical image segmentation
• Adversarial Networks for the Detection of Aggressive Prostate Cancer
• Detection of masses and architectural distortions in digital breast tomosynthesis: a publicly available dataset of 5,060 patients and a deep learning model
• Deep Semi Supervised Generative Learning for Automated PD-L1 Tumor Cell Scoring on NSCLC Tissue Needle Biopsies
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Abnormal Colon Polyp Image Synthesis Using Conditional Adversarial Networks for Improved Detection Performance
• Segmentation of Lungs in Chest X-Ray Image Using Generative Adversarial Networks
• Deep Structural Causal Models for Tractable Counterfactual Inference
• Learning to synthesise the ageing brain without longitudinal data
• Diffusion-GAN: Training GANs with Diffusion
• DC-AL GAN: Pseudoprogression and True Tumor Progression of Glioblastoma Multiform Image Classification Based on DCGAN and AlexNet
• MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks
• Generative Adversarial Networks for MR-CT Deformable Image Registration
• Privacy-Protective-GAN for Face De-identification
• Normalization of breast MRIs using Cycle-Consistent Generative Adversarial Networks
• Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging
• A New Cervical Cytology Dataset for Nucleus Detection and Image Classification (Cervix93) and Methods for Cervical Nucleus Detection
• Unsupervised Brain Anomaly Detection and Segmentation with Transformers
• On the Evaluation of Conditional GANs
• Generative adversarial networks and adversarial methods in biomedical image analysis
• Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
• ProstateGAN: Mitigating Data Bias via Prostate Diffusion Imaging Synthesis with Generative Adversarial Networks
• Estimating and Improving Fairness with Adversarial Learning
• Synthesizing lesions using contextual GANs improves breast cancer classification on mammograms
• Adversarial Training for Disease Prediction from Electronic Health Records with Missing Data
• Medical Image Generation using Generative Adversarial Networks
• mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesis
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation
• Training GANs with Stronger Augmentations via Contrastive Discriminator
• Multi-modal AsynDGAN: Learn From Distributed Medical Image Data without Sharing Private Information
• Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
• Implanting Synthetic Lesions for Improving Liver Lesion Segmentation in CT Exams
• Privacy-Net: An Adversarial Approach for Identity-Obfuscated Segmentation of Medical Images

Cited by in corpus (6)

• A large-scale multicenter breast cancer DCE-MRI benchmark dataset with expert segmentations
• medigan: a Python library of pretrained generative models for medical image synthesis
• High-resolution synthesis of high-density breast mammograms: Application to improved fairness in deep learning based mass detection
• Report of the Medical Image De-Identification (MIDI) Task Group -- Best Practices and Recommendations
• Pre- to Post-Contrast Breast MRI Synthesis for Enhanced Tumour Segmentation
• Fréchet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets