Transferring GANs: generating images from limited data

Abstract

Transferring the knowledge of pretrained networks to new domains by means of finetuning is a widely used practice for applications based on discriminative models. To the best of our knowledge this practice has not been studied within the context of generative deep networks. Therefore, we study domain adaptation applied to image generation with generative adversarial networks. We evaluate several aspects of domain adaptation, including the impact of target domain size, the relative distance between source and target domain, and the initialization of conditional GANs. Our results show that using knowledge from pretrained networks can shorten the convergence time and can significantly improve the quality of the generated images, especially when the target data is limited. We show that these conclusions can also be drawn for conditional GANs even when the pretrained model was trained without conditioning. Our results also suggest that density may be more important than diversity and a dataset with one or few densely sampled classes may be a better source model than more diverse datasets such as ImageNet or Places.

arXiv commentECCV2018-camera ready

References in corpus (18)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• A Learned Representation For Artistic Style
• Adversarially Learned Inference
• Invertible Conditional GANs for image editing
• A note on the evaluation of generative models
• cGANs with Projection Discriminator
• Improved Adversarial Systems for 3D Object Generation and Reconstruction
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Semi-supervised Conditional GANs
• Ensembles of Generative Adversarial Networks
• Class-Splitting Generative Adversarial Networks

Cited by in corpus (10)

• Recent advances and clinical applications of deep learning in medical image analysis
• Training Generative Adversarial Networks with Limited Data
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs
• Memory Replay GANs: learning to generate images from new categories without forgetting
• Few-Shot Adaptation of Generative Adversarial Networks
• Few-shot Image Generation via Cross-domain Correspondence
• Image Generation From Small Datasets via Batch Statistics Adaptation
• Semantics Disentangling for Text-to-Image Generation
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
• Effective Data Augmentation with Multi-Domain Learning GANs