Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection

Abstract

We present a flexible and high-performance framework, named Pyramid R-CNN, for two-stage 3D object detection from point clouds. Current approaches generally rely on the points or voxels of interest for RoI feature extraction on the second stage, but cannot effectively handle the sparsity and non-uniform distribution of those points, and this may result in failures in detecting objects that are far away. To resolve the problems, we propose a novel second-stage module, named pyramid RoI head, to adaptively learn the features from the sparse points of interest. The pyramid RoI head consists of three key components. Firstly, we propose the RoI-grid Pyramid, which mitigates the sparsity problem by extensively collecting points of interest for each RoI in a pyramid manner. Secondly, we propose RoI-grid Attention, a new operation that can encode richer information from sparse points by incorporating conventional attention-based and graph-based point operators into a unified formulation. Thirdly, we propose the Density-Aware Radius Prediction (DARP) module, which can adapt to different point density levels by dynamically adjusting the focusing range of RoIs. Combining the three components, our pyramid RoI head is robust to the sparse and imbalanced circumstances, and can be applied upon various 3D backbones to consistently boost the detection performance. Extensive experiments show that Pyramid R-CNN outperforms the state-of-the-art 3D detection models by a large margin on both the KITTI dataset and the Waymo Open dataset.

arXiv commentTo appear at ICCV 2021

References in corpus (20)

• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Cascade R-CNN: Delving into High Quality Object Detection
• Scalability in Perception for Autonomous Driving: Waymo Open Dataset
• Joint 3D Proposal Generation and Object Detection from View Aggregation
• One Million Scenes for Autonomous Driving: ONCE Dataset
• Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
• StarNet: Targeted Computation for Object Detection in Point Clouds
• AFDet: Anchor Free One Stage 3D Object Detection
• Center-based 3D Object Detection and Tracking
• Multi-View 3D Object Detection Network for Autonomous Driving
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud
• CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection
• Interpolated Convolutional Networks for 3D Point Cloud Understanding
• Pillar-based Object Detection for Autonomous Driving
• PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
• Patch Refinement -- Localized 3D Object Detection
• HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection
• Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection
• Multi-Task Multi-Sensor Fusion for 3D Object Detection