On Leveraging Pretrained GANs for Generation with Limited Data

Abstract

Recent work has shown generative adversarial networks (GANs) can generate highly realistic images, that are often indistinguishable (by humans) from real images. Most images so generated are not contained in the training dataset, suggesting potential for augmenting training sets with GAN-generated data. While this scenario is of particular relevance when there are limited data available, there is still the issue of training the GAN itself based on that limited data. To facilitate this, we leverage existing GAN models pretrained on large-scale datasets (like ImageNet) to introduce additional knowledge (which may not exist within the limited data), following the concept of transfer learning. Demonstrated by natural-image generation, we reveal that low-level filters (those close to observations) of both the generator and discriminator of pretrained GANs can be transferred to facilitate generation in a perceptually-distinct target domain with limited training data. To further adapt the transferred filters to the target domain, we propose adaptive filter modulation (AdaFM). An extensive set of experiments is presented to demonstrate the effectiveness of the proposed techniques on generation with limited data.

arXiv commentAccepted at ICML 2020

References in corpus (26)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• How transferable are features in deep neural networks?
• Spectral Normalization for Generative Adversarial Networks
• Learning Transferable Features with Deep Adaptation Networks
• Self-Attention Generative Adversarial Networks
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Generative Adversarial Network in Medical Imaging: A Review
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Which Training Methods for GANs do actually Converge?
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
• MaskGAN: Better Text Generation via Filling in the______
• GANSynth: Adversarial Neural Audio Synthesis
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Adversarial Ranking for Language Generation
• Learning Implicit Brain MRI Manifolds with Deep Learning
• Video-to-Video Synthesis
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
• Towards generative adversarial networks as a new paradigm for radiology education
• Probability density distillation with generative adversarial networks for high-quality parallel waveform generation
• Generative Knowledge Transfer for Neural Language Models
• Open Event Extraction from Online Text using a Generative Adversarial Network
• GO Gradient for Expectation-Based Objectives

Cited by in corpus (5)

• One-Shot Generative Domain Adaptation
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
• GAN Memory with No Forgetting
• Transfer Learning with Pre-trained Conditional Generative Models
• On Leveraging Unlabeled Data for Concurrent Positive-Unlabeled Classification and Robust Generation