U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

Abstract

We propose a novel method for unsupervised image-to-image translation, which incorporates a new attention module and a new learnable normalization function in an end-to-end manner. The attention module guides our model to focus on more important regions distinguishing between source and target domains based on the attention map obtained by the auxiliary classifier. Unlike previous attention-based method which cannot handle the geometric changes between domains, our model can translate both images requiring holistic changes and images requiring large shape changes. Moreover, our new AdaLIN (Adaptive Layer-Instance Normalization) function helps our attention-guided model to flexibly control the amount of change in shape and texture by learned parameters depending on datasets. Experimental results show the superiority of the proposed method compared to the existing state-of-the-art models with a fixed network architecture and hyper-parameters. Our code and datasets are available at https://github.com/taki0112/UGATIT or https://github.com/znxlwm/UGATIT-pytorch.

arXiv commentAccepted to ICLR 2020

References in corpus (8)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• Energy-based Generative Adversarial Network
• A Learned Representation For Artistic Style
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Multimodal Unsupervised Image-to-Image Translation
• Unsupervised Attention-guided Image to Image Translation

Cited by in corpus (33)

• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
• A New Dataset, Poisson GAN and AquaNet for Underwater Object Grabbing
• Scaling Distributed Machine Learning with In-Network Aggregation
• VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers
• Facial-Sketch Synthesis: A New Challenge
• Attention2AngioGAN: Synthesizing Fluorescein Angiography from Retinal Fundus Images using Generative Adversarial Networks
• Feature Quantization Improves GAN Training
• A Framework for Generalizing Critical Heat Flux Detection Models Using Unsupervised Image-to-Image Translation
• Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation
• AdvHaze: Adversarial Haze Attack
• Image-to-Image Translation: Methods and Applications
• Unsupervised-learning-based method for chest MRI-CT transformation using structure constrained unsupervised generative attention networks
• Remote Sensing Image Translation via Style-Based Recalibration Module and Improved Style Discriminator
• Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translation
• Hypercomplex Image-to-Image Translation
• Group Equivariant Generative Adversarial Networks
• Domain-Scalable Unpaired Image Translation via Latent Space Anchoring
• CAP-GAN: Towards Adversarial Robustness with Cycle-consistent Attentional Purification
• Unsupervised Sketch-to-Photo Synthesis
• Unsupervised Shadow Removal Using Target Consistency Generative Adversarial Network
• Unpaired Image-to-Image Translation using Adversarial Consistency Loss
• WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
• Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation
• Domain-Specific Mappings for Generative Adversarial Style Transfer
• PREGAN: Pose Randomization and Estimation for Weakly Paired Image Style Translation
• Unpaired Image-to-Image Translation via Latent Energy Transport
• DTGAN: Dual Attention Generative Adversarial Networks for Text-to-Image Generation
• BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer
• Global and Local Alignment Networks for Unpaired Image-to-Image Translation
• Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
• UMLE: Unsupervised Multi-discriminator Network for Low Light Enhancement
• DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation
• TransferI2I: Transfer Learning for Image-to-Image Translation from Small Datasets