Instance Map based Image Synthesis with a Denoising Generative Adversarial Network

Abstract

Semantic layouts based Image synthesizing, which has benefited from the success of Generative Adversarial Network (GAN), has drawn much attention in these days. How to enhance the synthesis image equality while keeping the stochasticity of the GAN is still a challenge. We propose a novel denoising framework to handle this problem. The overlapped objects generation is another challenging task when synthesizing images from a semantic layout to a realistic RGB photo. To overcome this deficiency, we include a one-hot semantic label map to force the generator paying more attention on the overlapped objects generation. Furthermore, we improve the loss function of the discriminator by considering perturb loss and cascade layer loss to guide the generation process. We applied our methods on the Cityscapes, Facades and NYU datasets and demonstrate the image generation ability of our model.

10 pages, 16figures

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Residual Learning for Image Recognition
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Improved Training of Wasserstein GANs
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Energy-based Generative Adversarial Network
• Generative Visual Manipulation on the Natural Image Manifold
• Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
• Age Progression/Regression by Conditional Adversarial Autoencoder
• Language Generation with Recurrent Generative Adversarial Networks without Pre-training
• Super-Resolution with Deep Convolutional Sufficient Statistics
• Photographic Image Synthesis with Cascaded Refinement Networks
• Scribbler: Controlling Deep Image Synthesis with Sketch and Color