Translate the Facial Regions You Like Using Region-Wise Normalization

Abstract

Though GAN (Generative Adversarial Networks) based technique has greatly advanced the performance of image synthesis and face translation, only few works available in literature provide region based style encoding and translation. We propose in this paper a region-wise normalization framework, for region level face translation. While per-region style is encoded using available approach, we build a so called RIN (region-wise normalization) block to individually inject the styles into per-region feature maps and then fuse them for following convolution and upsampling. Both shape and texture of different regions can thus be translated to various target styles. A region matching loss has also been proposed to significantly reduce the inference between regions during the translation process. Extensive experiments on three publicly available datasets, i.e. Morph, RaFD and CelebAMask-HQ, suggest that our approach demonstrate a large improvement over state-of-the-art methods like StarGAN, SEAN and FUNIT. Our approach has further advantages in precise control of the regions to be translated. As a result, region level expression changes and step by step make up can be achieved. The video demo is available at https://youtu.be/ceRqsbzXAfk.

arXiv comment13 pages, 13 figures

References in corpus (9)

• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Generative Adversarial Text to Image Synthesis
• Which Training Methods for GANs do actually Converge?
• StarGAN v2: Diverse Image Synthesis for Multiple Domains
• Patch-Based Image Inpainting with Generative Adversarial Networks
• MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
• TileGAN: Synthesis of Large-Scale Non-Homogeneous Textures