Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation
arXiv:1904.06807
Abstract
Cross-view image translation is challenging because it involves images with drastically different views and severe deformation. In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (SelectionGAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map. The proposed SelectionGAN explicitly utilizes the semantic information and consists of two stages. In the first stage, the condition image and the target semantic map are fed into a cycled semantic-guided generation network to produce initial coarse results. In the second stage, we refine the initial results by using a multi-channel attention selection mechanism. Moreover, uncertainty maps automatically learned from attentions are used to guide the pixel loss for better network optimization. Extensive experiments on Dayton, CVUSA and Ego2Top datasets show that our model is able to generate significantly better results than the state-of-the-art methods. The source code, data and trained models are available at https://github.com/Ha0Tang/SelectionGAN.
arXiv comment20 pages, 16 figures, accepted to CVPR 2019 as an oral paper
References in corpus (10)
- Conditional Generative Adversarial Nets
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
- Semi-Supervised Learning with Generative Adversarial Networks
- Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
- Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
- Unsupervised Attention-guided Image to Image Translation
- Learning What and Where to Draw
- Whitening and Coloring batch transform for GANs
- Novel View Synthesis for Large-scale Scene using Adversarial Loss