GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Abstract

Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods for generating synthetic medical images using recently presented deep learning Generative Adversarial Networks (GANs). Furthermore, we show that generated medical images can be used for synthetic data augmentation, and improve the performance of CNN for medical image classification. Our novel method is demonstrated on a limited dataset of computed tomography (CT) images of 182 liver lesions (53 cysts, 64 metastases and 65 hemangiomas). We first exploit GAN architectures for synthesizing high quality liver lesion ROIs. Then we present a novel scheme for liver lesion classification using CNN. Finally, we train the CNN using classic data augmentation and our synthetic data augmentation and compare performance. In addition, we explore the quality of our synthesized examples using visualization and expert assessment. The classification performance using only classic data augmentation yielded 78.6% sensitivity and 88.4% specificity. By adding the synthetic data augmentation the results increased to 85.7% sensitivity and 92.4% specificity. We believe that this approach to synthetic data augmentation can generalize to other medical classification applications and thus support radiologists' efforts to improve diagnosis.

Preprint submitted to Neurocomputing

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• A Survey on Deep Learning in Medical Image Analysis
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Improved Techniques for Training GANs
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation
• Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
• Towards Adversarial Retinal Image Synthesis
• Virtual PET Images from CT Data Using Deep Convolutional Networks: Initial Results
• SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays
• Medical Image Synthesis with Context-Aware Generative Adversarial Networks

Cited by in corpus (136)

• Generative Adversarial Network in Medical Imaging: A Review
• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• Recent advances and clinical applications of deep learning in medical image analysis
• A Comprehensive Survey of Image Augmentation Techniques for Deep Learning
• ResViT: Residual vision transformers for multi-modal medical image synthesis
• On Data Augmentation for GAN Training
• Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Modality specific U-Net variants for biomedical image segmentation: A survey
• Adversarial Text-to-Image Synthesis: A Review
• Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification
• Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN
• Latent Variable Models in the Era of Industrial Big Data: Extension and Beyond
• The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness
• Simulating Brain Signals: Creating Synthetic EEG Data via Neural-Based Generative Models for Improved SSVEP Classification
• A survey of synthetic data augmentation methods in computer vision
• Automatic Ischemic Stroke Lesion Segmentation from Computed Tomography Perfusion Images by Image Synthesis and Attention-Based Deep Neural Networks
• Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation
• Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencoder
• EEG-GAN: Generative adversarial networks for electroencephalograhic (EEG) brain signals
• Recent Advances in Network-based Methods for Disease Gene Prediction
• Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
• Evaluation of Deep Convolutional Generative Adversarial Networks for data augmentation of chest X-ray images
• Leveraging Generative AI Models for Synthetic Data Generation in Healthcare: Balancing Research and Privacy
• SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
• Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis
• DropMessage: Unifying Random Dropping for Graph Neural Networks
• Generative adversarial network with object detector discriminator for enhanced defect detection on ultrasonic B-scans
• Memory-efficient GAN-based Domain Translation of High Resolution 3D Medical Images
• Synthetically Enhanced: Unveiling Synthetic Data's Potential in Medical Imaging Research
• On Leveraging Pretrained GANs for Generation with Limited Data
• Physical Knowledge Enhanced Deep Neural Network for Sea Surface Temperature Prediction
• Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge
• Jekyll: Attacking Medical Image Diagnostics using Deep Generative Models
• On generative models as the basis for digital twins
• Generative Image Translation for Data Augmentation of Bone Lesion Pathology
• Large-Scale Generative Data-Free Distillation
• Improved Segmentation and Detection Sensitivity of Diffusion-Weighted Brain Infarct Lesions with Synthetically Enhanced Deep Learning
• CAA-Net: Conditional Atrous CNNs with Attention for Explainable Device-robust Acoustic Scene Classification
• A Unified Multi-Phase CT Synthesis and Classification Framework for Kidney Cancer Diagnosis with Incomplete Data
• Data Augmentation and Classification of Sea-Land Clutter for Over-the-Horizon Radar Using AC-VAEGAN
• Synthesizing time-series wound prognosis factors from electronic medical records using generative adversarial networks
• Probing transfer learning with a model of synthetic correlated datasets
• Decompose to manipulate: Manipulable Object Synthesis in 3D Medical Images with Structured Image Decomposition
• Improving Performance in Colorectal Cancer Histology Decomposition using Deep and Ensemble Machine Learning
• 3D Tomographic Pattern Synthesis for Enhancing the Quantification of COVID-19
• ProstateGAN: Mitigating Data Bias via Prostate Diffusion Imaging Synthesis with Generative Adversarial Networks
• HSMix: Hard and Soft Mixing Data Augmentation for Medical Image Segmentation
• Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization
• General Purpose (GenP) Bioimage Ensemble of Handcrafted and Learned Features with Data Augmentation
• Performance of GAN-based augmentation for deep learning COVID-19 image classification
• Federated Learning for Smart Healthcare: A Survey
• A Self Supervised StyleGAN for Image Annotation and Classification with Extremely Limited Labels
• DARE: Data Augmented Relation Extraction with GPT-2
• Generative Adversarial Networks (GANs): What it can generate and What it cannot?
• Exploring the Efficacy of Base Data Augmentation Methods in Deep Learning-Based Radiograph Classification of Knee Joint Osteoarthritis
• Material Measurement Units for a Circular Economy: Foundations through a Review
• Automatic Diagnosis of Pneumothorax from Chest Radiographs: A Systematic Literature Review
• Inspector Gadget: A Data Programming-based Labeling System for Industrial Images
• DAISI: Database for AI Surgical Instruction
• Learning More with Less: GAN-based Medical Image Augmentation
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation
• Time-series Imputation and Prediction with Bi-Directional Generative Adversarial Networks
• Harvesting, Detecting, and Characterizing Liver Lesions from Large-scale Multi-phase CT Data via Deep Dynamic Texture Learning
• GANs 'N Lungs: improving pneumonia prediction
• Knowledge-based Radiation Treatment Planning: A Data-driven Method Survey
• General-to-Detailed GAN for Infrequent Class Medical Images
• SVM-Lattice: A Recognition & Evaluation Frame for Double-peaked Profiles
• EfficientSeg: An Efficient Semantic Segmentation Network
• MITNet: GAN Enhanced Magnetic Induction Tomography Based on Complex CNN
• Additional Look into GAN-based Augmentation for Deep Learning COVID-19 Image Classification
• HAGAN: Hybrid Augmented Generative Adversarial Network for Medical Image Synthesis
• Measuring the Effectiveness of Voice Conversion on Speaker Identification and Automatic Speech Recognition Systems
• Constrained Generative Adversarial Network Ensembles for Sharable Synthetic Data Generation
• Synthetic Augmentation and Feature-based Filtering for Improved Cervical Histopathology Image Classification
• Improving Global Adversarial Robustness Generalization With Adversarially Trained GAN
• Towards Better Long-range Time Series Forecasting using Generative Adversarial Networks
• Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation
• Synthetic Sample Selection via Reinforcement Learning
• Feeding the zombies: Synthesizing brain volumes using a 3D progressive growing GAN
• Synthesizing brain tumor images and annotations by combining progressive growing GAN and SPADE
• MRI to PET Cross-Modality Translation using Globally and Locally Aware GAN (GLA-GAN) for Multi-Modal Diagnosis of Alzheimer's Disease
• Diffusion-Weighted Magnetic Resonance Brain Images Generation with Generative Adversarial Networks and Variational Autoencoders: A Comparison Study
• A Deep Learning Approach to Private Data Sharing of Medical Images Using Conditional GANs
• On the Pitfalls of Learning with Limited Data: A Facial Expression Recognition Case Study
• GIQA: Generated Image Quality Assessment
• Robust Unsupervised Small Area Change Detection from SAR Imagery Using Deep Learning
• Selective Synthetic Augmentation with Quality Assurance
• Ebola Optimization Search Algorithm (EOSA): A new metaheuristic algorithm based on the propagation model of Ebola virus disease
• A multi-stage GAN for multi-organ chest X-ray image generation and segmentation
• Improving Astronomical Time-series Classification via Data Augmentation with Generative Adversarial Networks
• Co-supervised learning paradigm with conditional generative adversarial networks for sample-efficient classification
• Red-GAN: Attacking class imbalance via conditioned generation. Yet another perspective on medical image synthesis for skin lesion dermoscopy and brain tumor MRI
• Synthetic Epileptic Brain Activities Using Generative Adversarial Networks
• GAN Memory with No Forgetting
• ScarGAN: Chained Generative Adversarial Networks to Simulate Pathological Tissue on Cardiovascular MR Scans
• Modeling EEG data distribution with a Wasserstein Generative Adversarial Network to predict RSVP Events
• Practical License Plate Recognition in Unconstrained Surveillance Systems with Adversarial Super-Resolution
• FetalFlex: Anatomy-Guided Diffusion Model for Flexible Control on Fetal Ultrasound Image Synthesis
• X-ray2CTPA: Leveraging Diffusion Models to Enhance Pulmonary Embolism Classification
• When do GANs replicate? On the choice of dataset size
• Anatomical Data Augmentation via Fluid-based Image Registration
• Semi-supervised Semantic Segmentation of Prostate and Organs-at-Risk on 3D Pelvic CT Images
• Learning More with Less: Conditional PGGAN-based Data Augmentation for Brain Metastases Detection Using Highly-Rough Annotation on MR Images
• The Application of Deep Learning for Lymph Node Segmentation: A Systematic Review
• Liver Segmentation using Turbolift Learning for CT and Cone-beam C-arm Perfusion Imaging
• Image Augmentation Using a Task Guided Generative Adversarial Network for Age Estimation on Brain MRI
• Deep learning scheme for recovery of broadband microwave photonic receiving systems in transceivers without expert knowledge and system priors
• Exploring Frequency Domain Interpretation of Convolutional Neural Networks
• Automatic detection of impact craters on Al foils from the Stardust interstellar dust collector using convolutional neural networks
• Synthetic Augmentation for Anatomical Landmark Localization using DDPMs
• Bayesian analysis of the prevalence bias: learning and predicting from imbalanced data
• Generative Modeling for Multi-task Visual Learning
• Computer-Assisted Analysis of Biomedical Images
• Learning Non-Unique Segmentation with Reward-Penalty Dice Loss
• Controllable Data Augmentation Through Deep Relighting
• Cascade Watchdog: A Multi-tiered Adversarial Guard for Outlier Detection
• Teaching a GAN What Not to Learn
• PanDA: Panoptic Data Augmentation
• Discriminative Cross-Modal Data Augmentation for Medical Imaging Applications
• Personalized One-Shot Lipreading for an ALS Patient
• Bayesian Sampling Bias Correction: Training with the Right Loss Function
• Bulk Production Augmentation Towards Explainable Melanoma Diagnosis
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation
• On the Performance of Generative Adversarial Network (GAN) Variants: A Clinical Data Study
• Generative Synthetic Augmentation using Label-to-Image Translation for Nuclei Image Segmentation
• Synthetic Melanoma Image Generation and Evaluation Using Generative Adversarial Networks
• Regularising Deep Networks with Deep Generative Models
• Domain Adaptation for Rare Classes Augmented with Synthetic Samples
• Improving Tuberculosis (TB) Prediction using Synthetically Generated Computed Tomography (CT) Images
• Parallel Medical Imaging for Intelligent Medical Image Analysis: Concepts, Methods, and Applications
• Pathology-Aware Generative Adversarial Networks for Medical Image Augmentation
• CoopSubNet: Cooperating Subnetwork for Data-Driven Regularization of Deep Networks under Limited Training Budgets
• Feature2Mass: Visual Feature Processing in Latent Space for Realistic Labeled Mass Generation
• ScaleNet: An Unsupervised Representation Learning Method for Limited Information
• Anonymization of labeled TOF-MRA images for brain vessel segmentation using generative adversarial networks