GP-GAN: Towards Realistic High-Resolution Image Blending

Abstract

It is common but challenging to address high-resolution image blending in the automatic photo editing application. In this paper, we would like to focus on solving the problem of high-resolution image blending, where the composite images are provided. We propose a framework called Gaussian-Poisson Generative Adversarial Network (GP-GAN) to leverage the strengths of the classical gradient-based approach and Generative Adversarial Networks. To the best of our knowledge, it's the first work that explores the capability of GANs in high-resolution image blending task. Concretely, we propose Gaussian-Poisson Equation to formulate the high-resolution image blending problem, which is a joint optimization constrained by the gradient and color information. Inspired by the prior works, we obtain gradient information via applying gradient filters. To generate the color information, we propose a Blending GAN to learn the mapping between the composite images and the well-blended ones. Compared to the alternative methods, our approach can deliver high-resolution, realistic images with fewer bleedings and unpleasant artifacts. Experiments confirm that our approach achieves the state-of-the-art performance on Transient Attributes dataset. A user study on Amazon Mechanical Turk finds that the majority of workers are in favor of the proposed method.

arXiv commentAccepted by ACMMM 2019. The source code is available in https://github.com/wuhuikai/GP-GAN and there's also an online demo in http://wuhuikai.me/DeepJS

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Improved Techniques for Training GANs
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• DRAW: A Recurrent Neural Network For Image Generation
• Deep Image Harmonization
• Learning a Discriminative Model for the Perception of Realism in Composite Images

Cited by in corpus (17)

• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Improving the Harmony of the Composite Image by Spatial-Separated Attention Module
• Text as Neural Operator: Image Manipulation by Text Instruction
• Painterly Image Harmonization using Diffusion Model
• Generative Adversarial Networks (GANs): What it can generate and What it cannot?
• Effect of The Latent Structure on Clustering with GANs
• CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
• MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs
• Spatial Fusion GAN for Image Synthesis
• Adversarial Image Composition with Auxiliary Illumination
• GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
• Image Quality Assessment Techniques Show Improved Training and Evaluation of Autoencoder Generative Adversarial Networks
• Mode matching in GANs through latent space learning and inversion
• Spatiotemporal Data Fusion for Precipitation Nowcasting
• Recurrent Deconvolutional Generative Adversarial Networks with Application to Text Guided Video Generation
• Towards a Uniform Architecture for the Efficient Implementation of 2D and 3D Deconvolutional Neural Networks on FPGAs
• Deep Image Compositing