PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud

Abstract

In this paper, we propose PointRCNN for 3D object detection from raw point cloud. The whole framework is composed of two stages: stage-1 for the bottom-up 3D proposal generation and stage-2 for refining proposals in the canonical coordinates to obtain the final detection results. Instead of generating proposals from RGB image or projecting point cloud to bird's view or voxels as previous methods do, our stage-1 sub-network directly generates a small number of high-quality 3D proposals from point cloud in a bottom-up manner via segmenting the point cloud of the whole scene into foreground points and background. The stage-2 sub-network transforms the pooled points of each proposal to canonical coordinates to learn better local spatial features, which is combined with global semantic features of each point learned in stage-1 for accurate box refinement and confidence prediction. Extensive experiments on the 3D detection benchmark of KITTI dataset show that our proposed architecture outperforms state-of-the-art methods with remarkable margins by using only point cloud as input. The code is available at https://github.com/sshaoshuai/PointRCNN.

Accepted by CVPR 2019

References in corpus (10)

• YOLOv3: An Incremental Improvement
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• HDNET: Exploiting HD Maps for 3D Object Detection
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Joint 3D Proposal Generation and Object Detection from View Aggregation
• PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
• GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving
• Feature Intertwiner for Object Detection

Cited by in corpus (18)

• Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection
• 3DSSD: Point-based 3D Single Stage Object Detector
• Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
• GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving
• SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation
• LiDAR Spoofing Meets the New-Gen: Capability Improvements, Broken Assumptions, and New Attack Strategies
• DSGN: Deep Stereo Geometry Network for 3D Object Detection
• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection
• F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds
• Flexible Supervised Autonomy for Exploration in Subterranean Environments
• Learning Distilled Collaboration Graph for Multi-Agent Perception
• PASS3D: Precise and Accelerated Semantic Segmentation for 3D Point Cloud
• Multi-Sensor 3D Object Box Refinement for Autonomous Driving
• 3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction
• Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
• FVNet: 3D Front-View Proposal Generation for Real-Time Object Detection from Point Clouds
• Fast and Modular Autonomy Software for Autonomous Racing Vehicles
• Joint Spatial-Temporal Optimization for Stereo 3D Object Tracking