Stereo R-CNN based 3D Object Detection for Autonomous Driving

Abstract

We propose a 3D object detection method for autonomous driving by fully exploiting the sparse and dense, semantic and geometry information in stereo imagery. Our method, called Stereo R-CNN, extends Faster R-CNN for stereo inputs to simultaneously detect and associate object in left and right images. We add extra branches after stereo Region Proposal Network (RPN) to predict sparse keypoints, viewpoints, and object dimensions, which are combined with 2D left-right boxes to calculate a coarse 3D object bounding box. We then recover the accurate 3D bounding box by a region-based photometric alignment using left and right RoIs. Our method does not require depth input and 3D position supervision, however, outperforms all existing fully supervised image-based methods. Experiments on the challenging KITTI dataset show that our method outperforms the state-of-the-art stereo-based method by around 30% AP on both 3D detection and 3D localization tasks. Code has been released at https://github.com/HKUST-Aerial-Robotics/Stereo-RCNN.

Accepted by cvpr2019

References in corpus (7)

• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Joint 3D Proposal Generation and Object Detection from View Aggregation
• 3D Object Proposals using Stereo Imagery for Accurate Object Class Detection

Cited by in corpus (4)

• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection
• F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds
• 3D Bounding Box Estimation for Autonomous Vehicles by Cascaded Geometric Constraints and Depurated 2D Detections Using 3D Results
• Multi-Sensor 3D Object Box Refinement for Autonomous Driving