Adversarial Image Composition with Auxiliary Illumination

Abstract

Dealing with the inconsistency between a foreground object and a background image is a challenging task in high-fidelity image composition. State-of-the-art methods strive to harmonize the composed image by adapting the style of foreground objects to be compatible with the background image, whereas the potential shadow of foreground objects within the composed image which is critical to the composition realism is largely neglected. In this paper, we propose an Adversarial Image Composition Net (AIC-Net) that achieves realistic image composition by considering potential shadows that the foreground object projects in the composed image. A novel branched generation mechanism is proposed, which disentangles the generation of shadows and the transfer of foreground styles for optimal accomplishment of the two tasks simultaneously. A differentiable spatial transformation module is designed which bridges the local harmonization and the global harmonization to achieve their joint optimization effectively. Extensive experiments on pedestrian and car composition tasks show that the proposed AIC-Net achieves superior composition performance qualitatively and quantitatively.

Accepted to ACCV 2020 (Oral)

References in corpus (11)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Toward Multimodal Image-to-Image Translation
• UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking
• GP-GAN: Towards Realistic High-Resolution Image Blending
• Few-Shot Unsupervised Image-to-Image Translation
• Context-Aware Synthesis and Placement of Object Instances
• Scene Text Synthesis for Efficient and Effective Deep Network Training
• Towards Realistic 3D Embedding via View Alignment
• Deep Painterly Harmonization
• EMLight: Lighting Estimation via Spherical Distribution Approximation
• Spatial-Aware GAN for Unsupervised Person Re-identification