Generative Adversarial Frontal View to Bird View Synthesis

Abstract

Environment perception is an important task with great practical value and bird view is an essential part for creating panoramas of surrounding environment. Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging. To tackle this problem, we propose the BridgeGAN, i.e., a novel generative model for bird view synthesis. First, an intermediate view, i.e., homography view, is introduced to bridge the large gap. Next, conditioned on the three views (frontal view, homography view and bird view) in our task, a multi-GAN based model is proposed to learn the challenging cross-view translation. Extensive experiments conducted on a synthetic dataset have demonstrated that the images generated by our model are much better than those generated by existing methods, with more consistent global appearance and sharper details. Ablation studies and discussions show its reliability and robustness in some challenging cases.

arXiv commentAccepted to 3DV 2018; Codes are available at https://github.com/WERush/BridgeGAN

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Generative Adversarial Text to Image Synthesis
• Energy-based Generative Adversarial Network
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• DeepStereo: Learning to Predict New Views from the World's Imagery
• Scribbler: Controlling Deep Image Synthesis with Sketch and Color

Cited by in corpus (3)

• Cross-view Semantic Segmentation for Sensing Surroundings
• BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud
• Cross-View Image Synthesis with Deformable Convolution and Attention Mechanism