Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth

Abstract

Current geometry-based monocular 3D object detection models can efficiently detect objects by leveraging perspective geometry, but their performance is limited due to the absence of accurate depth information. Though this issue can be alleviated in a depth-based model where a depth estimation module is plugged to predict depth information before 3D box reasoning, the introduction of such module dramatically reduces the detection speed. Instead of training a costly depth estimator, we propose a rendering module to augment the training data by synthesizing images with virtual-depths. The rendering module takes as input the RGB image and its corresponding sparse depth image, outputs a variety of photo-realistic synthetic images, from which the detection model can learn more discriminative features to adapt to the depth changes of the objects. Besides, we introduce an auxiliary module to improve the detection model by jointly optimizing it through a depth estimation task. Both modules are working in the training time and no extra computation will be introduced to the detection model. Experiments show that by working with our proposed modules, a geometry-based model can represent the leading accuracy on the KITTI 3D detection benchmark.

arXiv comment10 pages, 8 figures

References in corpus (21)

• Deep Residual Learning for Image Recognition
• SGDR: Stochastic Gradient Descent with Warm Restarts
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Deep Convolutional Inverse Graphics Network
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• Generative Image Inpainting with Contextual Attention
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
• Stereo Magnification: Learning View Synthesis using Multiplane Images
• Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
• Orthographic Feature Transform for Monocular 3D Object Detection
• Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
• Multi-View 3D Object Detection Network for Autonomous Driving
• RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving
• MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localization
• SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation
• MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships
• 3D Bounding Box Estimation Using Deep Learning and Geometry
• Rethinking Pseudo-LiDAR Representation
• Extreme View Synthesis
• SurfConv: Bridging 3D and 2D Convolution for RGBD Images