Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
arXiv:1901.09280
Abstract
We present the first approach for 3D point-cloud to image translation based on conditional Generative Adversarial Networks (cGAN). The model handles multi-modal information sources from different domains, i.e. raw point-sets and images. The generator is capable of processing three conditions, whereas the point-cloud is encoded as raw point-set and camera projection. An image background patch is used as constraint to bias environmental texturing. A global approximation function within the generator is directly applied on the point-cloud (Point-Net). Hence, the representative learning model incorporates global 3D characteristics directly at the latent feature space. Conditions are used to bias the background and the viewpoint of the generated image. This opens up new ways in augmenting or texturing 3D data to aim the generation of fully individual images. We successfully evaluated our method on the Kitti and SunRGBD dataset with an outstanding object detection inception score.
References in corpus (15)
- Conditional Generative Adversarial Nets
- YOLOv3: An Incremental Improvement
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
- Generative Adversarial Text to Image Synthesis
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Improved Techniques for Training GANs
- ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
- VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
- Pyramid Scene Parsing Network
- Holistically-Nested Edge Detection
- MarrNet: 3D Shape Reconstruction via 2.5D Sketches
- Complex-YOLO: Real-time 3D Object Detection on Point Clouds
- Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
- 3D Fully Convolutional Network for Vehicle Detection in Point Cloud