Differentiable Augmentation for Data-Efficient GAN Training

Abstract

The performance of generative adversarial networks (GANs) heavily deteriorates given a limited amount of training data. This is mainly because the discriminator is memorizing the exact training set. To combat it, we propose Differentiable Augmentation (DiffAugment), a simple method that improves the data efficiency of GANs by imposing various types of differentiable augmentations on both real and fake samples. Previous attempts to directly augment the training data manipulate the distribution of real images, yielding little benefit; DiffAugment enables us to adopt the differentiable augmentation for the generated samples, effectively stabilizes training, and leads to better convergence. Experiments demonstrate consistent gains of our method over a variety of GAN architectures and loss functions for both unconditional and class-conditional generation. With DiffAugment, we achieve a state-of-the-art FID of 6.80 with an IS of 100.8 on ImageNet 128x128 and 2-4x reductions of FID given 1,000 images on FFHQ and LSUN. Furthermore, with only 20% training data, we can match the top performance on CIFAR-10 and CIFAR-100. Finally, our method can generate high-fidelity images using only 100 images without pre-training, while being on par with existing transfer learning algorithms. Code is available at https://github.com/mit-han-lab/data-efficient-gans.

arXiv commentNeurIPS 2020. Project: https://data-efficient-gans.mit.edu/ Code: https://github.com/mit-han-lab/data-efficient-gans

References in corpus (13)

• Denoising Diffusion Probabilistic Models
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Neural Architecture Search with Reinforcement Learning
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Self-Attention Generative Adversarial Networks
• Which Training Methods for GANs do actually Converge?
• AutoAugment: Learning Augmentation Policies from Data
• Amortised MAP Inference for Image Super-resolution
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
• Image Augmentations for GAN Training
• High-Fidelity Image Generation With Fewer Labels

Cited by in corpus (15)

• TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
• A Good Image Generator Is What You Need for High-Resolution Video Synthesis
• A Systematic Survey of Regularization and Normalization in GANs
• Improved Consistency Regularization for GANs
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• Differentiable Data Augmentation with Kornia
• Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data
• xAI-GAN: Enhancing Generative Adversarial Networks via Explainable AI Systems
• Scalable Balanced Training of Conditional Generative Adversarial Neural Networks on Image Data
• Learning Temporally Invariant and Localizable Features via Data Augmentation for Video Recognition
• On the Existence of Optimal Transport Gradient for Learning Generative Models
• Discriminative Cross-Modal Data Augmentation for Medical Imaging Applications
• Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection
• Robust Training Using Natural Transformation
• EXoN: EXplainable encoder Network