Training Generative Adversarial Networks with Limited Data

Abstract

Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes. The approach does not require changes to loss functions or network architectures, and is applicable both when training from scratch and when fine-tuning an existing GAN on another dataset. We demonstrate, on several datasets, that good results are now possible using only a few thousand training images, often matching StyleGAN2 results with an order of magnitude fewer images. We expect this to open up new application domains for GANs. We also find that the widely used CIFAR-10 is, in fact, a limited data benchmark, and improve the record FID from 5.59 to 2.42.

References in corpus (24)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Self-Attention Generative Adversarial Networks
• Generative Adversarial Network in Medical Imaging: A Review
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Which Training Methods for GANs do actually Converge?
• AutoAugment: Learning Augmentation Policies from Data
• Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
• cGANs with Projection Discriminator
• Improved Precision and Recall Metric for Assessing Generative Models
• Demystifying MMD GANs
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks
• Megapixel Size Image Creation using Generative Adversarial Networks
• Feature Quantization Improves GAN Training
• Improved Consistency Regularization for GANs
• Transferring GANs: generating images from limited data
• DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data
• cGANs with Multi-Hinge Loss
• Self-Supervised GANs via Auxiliary Rotation Loss
• Unsupervised multi-modal Styled Content Generation

Cited by in corpus (73)

• Denoising Diffusion Probabilistic Models
• Recent advances and clinical applications of deep learning in medical image analysis
• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Recent Progress in Transformer-based Medical Image Analysis
• A survey of synthetic data augmentation methods in computer vision
• Let's Wait Awhile: How Temporal Workload Shifting Can Reduce Carbon Emissions in the Cloud
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Structured Denoising Diffusion Models in Discrete State-Spaces
• A Good Image Generator Is What You Need for High-Resolution Video Synthesis
• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
• A Systematic Survey of Regularization and Normalization in GANs
• Machine Learning in NextG Networks via Generative Adversarial Networks
• FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
• Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey
• Evaluating the Performance of StyleGAN2-ADA on Medical Images
• Distinguishing Natural and Computer-Generated Images using Multi-Colorspace fused EfficientNet
• Improved Consistency Regularization for GANs
• Photoacoustic image synthesis with generative adversarial networks
• Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression
• CosSIF: Cosine similarity-based image filtering to overcome low inter-class variation in synthetic medical image datasets
• Everything is There in Latent Space: Attribute Editing and Attribute Style Manipulation by StyleGAN Latent Space Exploration
• Differentiable JPEG: The Devil is in the Details
• Latent Denoising Diffusion GAN: Faster sampling, Higher image quality
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• Boosting EfficientNets Ensemble Performance via Pseudo-Labels and Synthetic Images by pix2pixHD for Infection and Ischaemia Classification in Diabetic Foot Ulcers
• Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend
• Making Invisible Visible: Data-Driven Seismic Inversion with Spatio-temporally Constrained Data Augmentation
• Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images
• FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
• Performance of GAN-based augmentation for deep learning COVID-19 image classification
• Graph Generative Adversarial Networks for Sparse Data Generation in High Energy Physics
• Learning Stage-wise GANs for Whistle Extraction in Time-Frequency Spectrograms
• Diabetic Foot Ulcer Grand Challenge 2022 Summary
• Bayesian Image Reconstruction using Deep Generative Models
• MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs
• Triple Generative Adversarial Networks
• A Contrastive Learning Approach for Training Variational Autoencoder Priors
• Probabilistic Precision and Recall Towards Reliable Evaluation of Generative Models
• Stylized Face Sketch Extraction via Generative Prior with Limited Data
• Translation-Enhanced Multilingual Text-to-Image Generation
• Anycost GANs for Interactive Image Synthesis and Editing
• PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping
• Additional Look into GAN-based Augmentation for Deep Learning COVID-19 Image Classification
• A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications
• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling
• Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poems
• 3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images
• Defending Medical Image Diagnostics against Privacy Attacks using Generative Methods
• ID-Booth: Identity-consistent Face Generation with Diffusion Models
• On the Pitfalls of Learning with Limited Data: A Facial Expression Recognition Case Study
• Synthetic data shuffling accelerates the convergence of federated learning under data heterogeneity
• Data Augmentation and Transfer Learning Approaches Applied to Facial Expressions Recognition
• Learning Temporally Invariant and Localizable Features via Data Augmentation for Video Recognition
• Transfer Learning with Pre-trained Conditional Generative Models
• Omni-GAN: On the Secrets of cGANs and Beyond
• Inclusive normalization of face images to passport format
• A Regularized Conditional GAN for Posterior Sampling in Image Recovery Problems
• LATENTPATCH: A Non-Parametric Approach for Face Generation and Editing
• Enhancing Stability in Training Conditional Generative Adversarial Networks via Selective Data Matching
• DACov: A Deeper Analysis of Data Augmentation on the Computed Tomography Segmentation Problem
• Reconstructing Interpretable Features in Computational Super-Resolution microscopy via Regularized Latent Search
• A Matrix Autoencoder Framework to Align the Functional and Structural Connectivity Manifolds as Guided by Behavioral Phenotypes
• Functional Space Analysis of Local GAN Convergence
• Adapting to Unseen Vendor Domains for MRI Lesion Segmentation
• Exploring constraints on CycleGAN-based CBCT enhancement for adaptive radiotherapy
• CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation
• SAT3D: Image-driven Semantic Attribute Transfer in 3D
• Generative Adversarial Networks for Astronomical Images Generation
• Leveraging Domain Adaptation for Low-Resource Geospatial Machine Learning
• deepNIR: Datasets for generating synthetic NIR images and improved fruit detection system using deep learning techniques
• SurvODE: Extrapolating Gene Expression Distribution for Early Cancer Identification