DSGN: Deep Stereo Geometry Network for 3D Object Detection

Abstract

Most state-of-the-art 3D object detectors heavily rely on LiDAR sensors because there is a large performance gap between image-based and LiDAR-based methods. It is caused by the way to form representation for the prediction in 3D scenarios. Our method, called Deep Stereo Geometry Network (DSGN), significantly reduces this gap by detecting 3D objects on a differentiable volumetric representation -- 3D geometric volume, which effectively encodes 3D geometric structure for 3D regular space. With this representation, we learn depth information and semantic cues simultaneously. For the first time, we provide a simple and effective one-stage stereo-based 3D detection pipeline that jointly estimates the depth and detects 3D objects in an end-to-end learning manner. Our approach outperforms previous stereo-based 3D detectors (about 10 higher in terms of AP) and even achieves comparable performance with several LiDAR-based methods on the KITTI 3D object detection leaderboard. Our code is publicly available at https://github.com/chenyilun95/DSGN.

arXiv commentAccepted by CVPR 2020 (Camera Ready)

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ShapeNet: An Information-Rich 3D Model Repository
• FCOS: Fully Convolutional One-Stage Object Detection
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Learning a Multi-View Stereo Machine
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
• Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
• Orthographic Feature Transform for Monocular 3D Object Detection
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud
• 3D Object Proposals using Stereo Imagery for Accurate Object Class Detection
• Triangulation Learning Network: from Monocular to Stereo 3D Object Detection
• Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving
• Point-Based Multi-View Stereo Network

Cited by in corpus (5)

• Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds
• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection
• Confidence Guided Stereo 3D Object Detection with Split Depth Estimation
• Dynamic Edge Weights in Graph Neural Networks for 3D Object Detection
• M3DSSD: Monocular 3D Single Stage Object Detector