Generative Adversarial Network in Medical Imaging: A Review

Abstract

Generative adversarial networks have gained a lot of attention in the computer vision community due to their capability of data generation without explicitly modelling the probability density function. The adversarial loss brought by the discriminator provides a clever way of incorporating unlabeled samples into training and imposing higher order consistency. This has proven to be useful in many cases, such as domain adaptation, data augmentation, and image-to-image translation. These properties have attracted researchers in the medical imaging community, and we have seen rapid adoption in many traditional and novel applications, such as image reconstruction, segmentation, detection, classification, and cross-modality synthesis. Based on our observations, this trend will continue and we therefore conducted a review of recent advances in medical imaging using the adversarial training scheme with the hope of benefiting researchers interested in this technique.

24 pages; v4; added missing references from before Jan 1st 2019; accepted to MedIA

References in corpus (103)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• A Survey on Deep Learning in Medical Image Analysis
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Networks: An Overview
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Energy-based Generative Adversarial Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• AutoAugment: Learning Augmentation Policies from Data
• Adversarial Feature Learning
• Adversarially Learned Inference
• Autoencoding beyond pixels using a learned similarity metric
• Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Wasserstein GAN
• On the Automatic Generation of Medical Imaging Reports
• 3D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Network
• cGANs with Projection Discriminator
• Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)
• GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
• MaskGAN: Better Text Generation via Filling in the______
• Sharpness-aware Low dose CT denoising using conditional generative adversarial network
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Distribution Matching Losses Can Hallucinate Features in Medical Image Translation
• Structure-sensitive Multi-scale Deep Neural Network for Low-Dose CT Denoising
• Generative Adversarial Network for Medical Images (MI-GAN)
• Semantic Image Synthesis with Spatially-Adaptive Normalization
• Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy
• Cycle Consistent Adversarial Denoising Network for Multiphase Coronary CT Angiography
• Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training
• Brain MRI Super Resolution Using 3D Deep Densely Connected Neural Networks
• Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation
• Learning What and Where to Draw
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks
• TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
• Learning Implicit Brain MRI Manifolds with Deep Learning
• Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
• Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning
• Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders
• Adversarial Networks for the Detection of Aggressive Prostate Cancer
• Synthetic Medical Images from Dual Generative Adversarial Networks
• Unsupervised Learning for Cell-level Visual Representation in Histopathology Images with Generative Adversarial Networks
• CT-GAN: Malicious Tampering of 3D Medical Imagery using Deep Learning
• Towards Diverse and Natural Image Descriptions via a Conditional GAN
• Generative Adversarial Networks for Image-to-Image Translation on Multi-Contrast MR Images - A Comparison of CycleGAN and UNIT
• Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks
• Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
• Towards Adversarial Retinal Image Synthesis
• Adversarial Inpainting of Medical Image Modalities
• Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy
• Automated Treatment Planning in Radiation Therapy using Generative Adversarial Networks
• Unsupervised Histopathology Image Synthesis
• Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network
• Refacing: reconstructing anonymized facial features using GANs
• High-resolution medical image synthesis using progressively grown generative adversarial networks
• Learning Myelin Content in Multiple Sclerosis from Multimodal MRI through Adversarial Training
• Generative Adversarial Networks for MR-CT Deformable Image Registration
• Brain MRI super-resolution using 3D generative adversarial networks
• MelanoGANs: High Resolution Skin Lesion Synthesis with GANs
• High-Resolution Mammogram Synthesis using Progressive Generative Adversarial Networks
• Unsupervised and semi-supervised learning with Categorical Generative Adversarial Networks assisted by Wasserstein distance for dermoscopy image Classification
• Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution
• Retrospective correction of Rigid and Non-Rigid MR motion artifacts using GANs
• Conditional Adversarial Network for Semantic Segmentation of Brain Tumor
• Adversarial Sparse-View CBCT Artifact Reduction
• Text to Image Synthesis Using Generative Adversarial Networks
• Semi-Supervised Deep Learning for Abnormality Classification in Retinal Images
• SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays
• Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs)
• X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks
• Adversarial training and dilated convolutions for brain MRI segmentation
• Conditional Generative Refinement Adversarial Networks for Unbalanced Medical Image Semantic Segmentation
• Towards generative adversarial networks as a new paradigm for radiology education
• Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data size
• MRI Cross-Modality NeuroImage-to-NeuroImage Translation
• Synthesizing Filamentary Structured Images with GANs
• ProstateGAN: Mitigating Data Bias via Prostate Diffusion Imaging Synthesis with Generative Adversarial Networks
• Automatic Liver Segmentation Using an Adversarial Image-to-Image Network
• An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection
• Unpaired Brain MR-to-CT Synthesis using a Structure-Constrained CycleGAN
• Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-scale Lesion Database
• Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images
• Learning Bone Suppression from Dual Energy Chest X-rays using Adversarial Networks
• Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI
• Retinal Vessel Segmentation under Extreme Low Annotation: A Generative Adversarial Network Approach
• Stabilizing Adversarial Nets With Prediction Methods
• Deep CT to MR Synthesis using Paired and Unpaired Data
• Which Contrast Does Matter? Towards a Deep Understanding of MR Contrast using Collaborative GAN
• Generating Highly Realistic Images of Skin Lesions with GANs
• Denoising of 3-D Magnetic Resonance Images Using a Residual Encoder-Decoder Wasserstein Generative Adversarial Network
• Generative Spatiotemporal Modeling Of Neutrophil Behavior

Cited by in corpus (162)

• U-Net and its variants for medical image segmentation: theory and applications
• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• Recent advances and clinical applications of deep learning in medical image analysis
• Training Generative Adversarial Networks with Limited Data
• All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda
• Deep neural network models for computational histopathology: A survey
• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis
• ResViT: Residual vision transformers for multi-modal medical image synthesis
• Deep Learning for Chest X-ray Analysis: A Survey
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
• Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review
• Deep learning-based synthetic-CT generation in radiotherapy and PET: a review
• A Survey on Deep Learning for Skin Lesion Segmentation
• Medical Image Registration Using Deep Neural Networks: A Comprehensive Review
• An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works
• Generative Adversarial Networks (GANs) in Networking: A Comprehensive Survey & Evaluation
• Generative Adversarial Networks for Spatio-temporal Data: A Survey
• DR-GAN: Conditional Generative Adversarial Network for Fine-Grained Lesion Synthesis on Diabetic Retinopathy Images
• Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis
• Unsupervised Medical Image Translation Using Cycle-MedGAN
• Image to Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography
• R2U++: A Multiscale Recurrent Residual U-Net with Dense Skip Connections for Medical Image Segmentation
• CT-GAN: Malicious Tampering of 3D Medical Imagery using Deep Learning
• Applications of Generative Adversarial Networks in Neuroimaging and Clinical Neuroscience
• Melanoma Detection using Adversarial Training and Deep Transfer Learning
• Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencoder
• Is attention all you need in medical image analysis? A review
• Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study
• Deep and Statistical Learning in Biomedical Imaging: State of the Art in 3D MRI Brain Tumor Segmentation
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
• Evaluation of Deep Convolutional Generative Adversarial Networks for data augmentation of chest X-ray images
• Similarity and Quality Metrics for MR Image-To-Image Translation
• A Systematic Survey of Regularization and Normalization in GANs
• SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation
• On Leveraging Pretrained GANs for Generation with Limited Data
• Refacing: reconstructing anonymized facial features using GANs
• Three-dimensional Generative Adversarial Nets for Unsupervised Metal Artifact Reduction
• medigan: a Python library of pretrained generative models for medical image synthesis
• Generative adversarial networks in time series: A survey and taxonomy
• FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
• Evaluating the Clinical Realism of Synthetic Chest X-Rays Generated Using Progressively Growing GANs
• Normalization of breast MRIs using Cycle-Consistent Generative Adversarial Networks
• Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation
• Deep Learning for Neuroimaging-based Diagnosis and Rehabilitation of Autism Spectrum Disorder: A Review
• Jekyll: Attacking Medical Image Diagnostics using Deep Generative Models
• A Survey on Deep Learning of Small Sample in Biomedical Image Analysis
• Conditional Generation of Medical Images via Disentangled Adversarial Inference
• Generation of microbial colonies dataset with deep learning style transfer
• Semi-Supervised Deep Learning for Abnormality Classification in Retinal Images
• Cephalogram Synthesis and Landmark Detection in Dental Cone-Beam CT Systems
• Accelerated and Quantitative 3D Semisolid MT/CEST Imaging using a Generative Adversarial Network (GAN-CEST)
• On the capacity of deep generative networks for approximating distributions
• Combating COVID-19 using Generative Adversarial Networks and Artificial Intelligence for Medical Images: A Scoping Review
• Using Conditional Generative Adversarial Networks to Reduce the Effects of Latency in Robotic Telesurgery
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• Generative Models Improve Radiomics Reproducibility in Low Dose CTs: A Simulation Study
• XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms
• Exploiting the Generative Adversarial Network Approach to Create a Synthetic Topography Corneal Image
• Classification of lung cancer subtypes on CT images with synthetic pathological priors
• Three dimensional blind image deconvolution for fluorescence microscopy using generative adversarial networks
• Deep Learning in Mining Biological Data
• Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy
• In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
• An error analysis of generative adversarial networks for learning distributions
• CIA-Net: Robust Nuclei Instance Segmentation with Contour-aware Information Aggregation
• GANravel: User-Driven Direction Disentanglement in Generative Adversarial Networks
• Physics-assisted Generative Adversarial Network for X-Ray Tomography
• Reconstructing undersampled photoacoustic microscopy images using deep learning
• Bridging the gap between paired and unpaired medical image translation
• Non-Parametric Data-Driven Background Modelling using Conditional Probabilities
• Deep learning-based conditional inpainting for restoration of artifact-affected 4D CT images
• Generalizable synthetic MRI with physics-informed convolutional networks
• Synthetic Observational Health Data with GANs: from slow adoption to a boom in medical research and ultimately digital twins?
• MRI super-resolution reconstruction using efficient diffusion probabilistic model with residual shifting
• Medical Image Generation using Generative Adversarial Networks
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation
• This Intestine Does Not Exist: Multiscale Residual Variational Autoencoder for Realistic Wireless Capsule Endoscopy Image Generation
• 3D Brain and Heart Volume Generative Models: A Survey
• Handwritten Word Recognition using Deep Learning Approach: A Novel Way of Generating Handwritten Words
• Swap-Free Fat-Water Separation in Dixon MRI using Conditional Generative Adversarial Networks
• A Transfer Learning Approach for Automated Segmentation of Prostate Whole Gland and Transition Zone in Diffusion Weighted MRI
• privGAN: Protecting GANs from membership inference attacks at low cost
• Artificial Intelligence-Based Image Classification for Diagnosis of Skin Cancer: Challenges and Opportunities
• GAN-GA: A Generative Model based on Genetic Algorithm for Medical Image Generation
• Brain Tumor Synthetic Data Generation with Adaptive StyleGANs
• A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels
• Realistic Ultrasound Image Synthesis for Improved Classification of Liver Disease
• Generating fMRI volumes from T1-weighted volumes using 3D CycleGAN
• Data Augmentation For Medical MR Image Using Generative Adversarial Networks
• MACE: A Flexible Framework for Membership Privacy Estimation in Generative Models
• Anatomy and Physiology of Artificial Intelligence in PET Imaging
• Accurate generation of stochastic dynamics based on multi-model Generative Adversarial Networks
• Additional Look into GAN-based Augmentation for Deep Learning COVID-19 Image Classification
• Constrained Generative Adversarial Network Ensembles for Sharable Synthetic Data Generation
• Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts
• Deep Learning in Multi-organ Segmentation
• Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis
• Feeding the zombies: Synthesizing brain volumes using a 3D progressive growing GAN
• Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis
• Pathology Synthesis of 3D-Consistent Cardiac MR Images using 2D VAEs and GANs
• Tensorizing GAN with High-Order Pooling for Alzheimer's Disease Assessment
• HealthiVert-GAN: A Novel Framework of Pseudo-Healthy Vertebral Image Synthesis for Interpretable Compression Fracture Grading
• Ebola Optimization Search Algorithm (EOSA): A new metaheuristic algorithm based on the propagation model of Ebola virus disease
• Score-Guided Generative Adversarial Networks
• CNNs and GANs in MRI-based cross-modality medical image estimation
• A Deep Learning Approach to Private Data Sharing of Medical Images Using Conditional GANs
• Systematic Review and Meta-analysis of AI-driven MRI Motion Artifact Detection and Correction
• No Surprises: Training Robust Lung Nodule Detection for Low-Dose CT Scans by Augmenting with Adversarial Attacks
• Deep-learning Image Reconstruction for Real-time Photoacoustic System
• Data Consistent CT Reconstruction from Insufficient Data with Learned Prior Images
• Red-GAN: Attacking class imbalance via conditioned generation. Yet another perspective on medical image synthesis for skin lesion dermoscopy and brain tumor MRI
• Interpretability of Uncertainty: Exploring Cortical Lesion Segmentation in Multiple Sclerosis
• Learning from Data-Rich Problems: A Case Study on Genetic Variant Calling
• Implanting Synthetic Lesions for Improving Liver Lesion Segmentation in CT Exams
• Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Images
• MeDaS: An open-source platform as service to help break the walls between medicine and informatics
• Tumor Synthesis conditioned on Radiomics
• Cross-domain Denoising for Low-dose Multi-frame Spiral Computed Tomography
• Fast refacing of MR images with a generative neural network lowers re-identification risk and preserves volumetric consistency
• Fréchet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets
• Observer study-based evaluation of a stochastic and physics-based method to generate oncological PET images
• Sample Amplification: Increasing Dataset Size even when Learning is Impossible
• A versatile anomaly detection method for medical images with a flow-based generative model in semi-supervision setting
• CT Image Synthesis Using Weakly Supervised Segmentation and Geometric Inter-Label Relations For COVID Image Analysis
• Chest CT Super-resolution and Domain-adaptation using Memory-efficient 3D Reversible GANs
• The use of Generative Adversarial Networks to characterise new physics in multi-lepton final states at the LHC
• One-class Steel Detector Using Patch GAN Discriminator for Visualising Anomalous Feature Map
• Using Generative Models for Pediatric wbMRI
• Structure Preserving Stain Normalization of Histopathology Images Using Self-Supervised Semantic Guidance
• Artificial Intelligence for Pediatric Ophthalmology
• Cascade Watchdog: A Multi-tiered Adversarial Guard for Outlier Detection
• GAN decoder on a quantum toric code for noise-robust quantum teleportation
• Deep infant brain segmentation from multi-contrast MRI
• Ocular Disease Classification Using CNN with Deep Convolutional Generative Adversarial Network
• Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging
• Artificial Intelligence in Quantitative Ultrasound Imaging: A Review
• Improving learnability of neural networks: adding supplementary axes to disentangle data representation
• Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction
• Game of GANs: Game-Theoretical Models for Generative Adversarial Networks
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Multi-stage domain adversarial style reconstruction for cytopathological image stain normalization
• Scalable Modeling of Spatiotemporal Data using the Variational Autoencoder: an Application in Glaucoma
• Fast Mesh Data Augmentation via Chebyshev Polynomial of Spectral filtering
• One Representative-Shot Learning Using a Population-Driven Template with Application to Brain Connectivity Classification and Evolution Prediction
• Spatial light interference microscopy (SLIM): principle and applications to biomedicine
• Pathology-Aware Generative Adversarial Networks for Medical Image Augmentation
• Assessing Automated Machine Learning service to detect COVID-19 from X-Ray and CT images: A Real-time Smartphone Application case study
• Decoupling Shape and Density for Liver Lesion Synthesis Using Conditional Generative Adversarial Networks
• Dynamic Flows on Curved Space Generated by Labeled Data
• Unsupervised Metal Artifact Reduction in Dental CBCT using Fine-tuned Cycle-Consistent Adversarial Networks
• Anonymization of labeled TOF-MRA images for brain vessel segmentation using generative adversarial networks
• A Rigid Registration Method in TEVAR
• Structural MRI Synthesis for Alzheimer's Disease via Conditional Diffusion on Anatomical Masks
• Performance Analysis and Optimization of 3D Generative Diffusion Models across GPU Architectures
• Where is the disease? Semi-supervised pseudo-normality synthesis from an abnormal image
• Generative Model Adversarial Training for Deep Compressed Sensing
• Image Translation for Medical Image Generation -- Ischemic Stroke Lesions
• Enhanced Magnetic Resonance Image Synthesis with Contrast-Aware Generative Adversarial Networks
• Generative Synthetic Augmentation using Label-to-Image Translation for Nuclei Image Segmentation
• Learning of Inter-Label Geometric Relationships Using Self-Supervised Learning: Application To Gleason Grade Segmentation