MU-GAN: Facial Attribute Editing based on Multi-attention Mechanism

Abstract

Facial attribute editing has mainly two objectives: 1) translating image from a source domain to a target one, and 2) only changing the facial regions related to a target attribute and preserving the attribute-excluding details. In this work, we propose a Multi-attention U-Net-based Generative Adversarial Network (MU-GAN). First, we replace a classic convolutional encoder-decoder with a symmetric U-Net-like structure in a generator, and then apply an additive attention mechanism to build attention-based U-Net connections for adaptively transferring encoder representations to complement a decoder with attribute-excluding detail and enhance attribute editing ability. Second, a self-attention mechanism is incorporated into convolutional layers for modeling long-range and multi-level dependencies across image regions. experimental results indicate that our method is capable of balancing attribute editing ability and details preservation ability, and can decouple the correlation among attributes. It outperforms the state-of-the-art methods in terms of attribute manipulation accuracy and image quality.

arXiv comment12 pages, 10 figures

References in corpus (10)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Invertible Conditional GANs for image editing
• Semi-Supervised Learning with Generative Adversarial Networks
• Residual Networks of Residual Networks: Multilevel Residual Networks
• Unsupervised Domain Adaptation with Adversarial Residual Transform Networks
• Unsupervised Image-to-Image Translation with Generative Adversarial Networks