Style-Restricted GAN: Multi-Modal Translation with Style Restriction Using Generative Adversarial Networks

Abstract

Unpaired image-to-image translation using Generative Adversarial Networks (GAN) is successful in converting images among multiple domains. Moreover, recent studies have shown a way to diversify the outputs of the generator. However, since there are no restrictions on how the generator diversifies the results, it is likely to translate some unexpected features. In this paper, we propose Style-Restricted GAN (SRGAN) to demonstrate the importance of controlling the encoded features used in style diversifying process. More specifically, instead of KL divergence loss, we adopt three new losses to restrict the distribution of the encoded features: batch KL divergence loss, correlation loss, and histogram imitation loss. Further, the encoder is pre-trained with classification tasks before being used in translation process. The study reports quantitative as well as qualitative results with Precision, Recall, Density, and Coverage. The proposed three losses lead to the enhancement of the level of diversity compared to the conventional KL loss. In particular, SRGAN is found to be successful in translating with higher diversity and without changing the class-unrelated features in the CelebA face dataset. To conclude, the importance of the encoded features being well-regulated was proven with two experiments. Our implementation is available at https://github.com/shinshoji01/Style-Restricted_GAN.

arXiv comment20 pages, 11 figures, 6 tables; Our implementation is available at https://github.com/shinshoji01/Style-Restricted_GAN

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Deep Residual Learning for Image Recognition
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Toward Multimodal Image-to-Image Translation
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• Unrolled Generative Adversarial Networks
• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• Scribbler: Controlling Deep Image Synthesis with Sketch and Color
• Multi-Mapping Image-to-Image Translation with Central Biasing Normalization