Image Generation From Small Datasets via Batch Statistics Adaptation

Abstract

Thanks to the recent development of deep generative models, it is becoming possible to generate high-quality images with both fidelity and diversity. However, the training of such generative models requires a large dataset. To reduce the amount of data required, we propose a new method for transferring prior knowledge of the pre-trained generator, which is trained with a large dataset, to a small dataset in a different domain. Using such prior knowledge, the model can generate images leveraging some common sense that cannot be acquired from a small dataset. In this work, we propose a novel method focusing on the parameters for batch statistics, scale and shift, of the hidden layers in the generator. By training only these parameters in a supervised manner, we achieved stable training of the generator, and our method can generate higher quality images compared to previous methods without collapsing, even when the dataset is small (~100). Our results show that the diversity of the filters acquired in the pre-trained generator is important for the performance on the target domain. Our method makes it possible to add a new class or domain to a pre-trained generator without disturbing the performance on the original domain.

arXiv commentICCV 2019

References in corpus (18)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Learning Transferable Features with Deep Adaptation Networks
• Self-Attention Generative Adversarial Networks
• Pixel Recurrent Neural Networks
• Semi-Supervised Learning with Deep Generative Models
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• A Learned Representation For Artistic Style
• Domain Separation Networks
• Adversarial Discriminative Domain Adaptation
• Revisiting Batch Normalization For Practical Domain Adaptation
• IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis
• Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
• Transferring GANs: generating images from limited data
• Generative Knowledge Transfer for Neural Language Models

Cited by in corpus (3)

• Group Equivariant Generative Adversarial Networks
• Selective Synthetic Augmentation with Quality Assurance
• Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation