PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Abstract

We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds. Our proposed method deeply integrates both 3D voxel Convolutional Neural Network (CNN) and PointNet-based set abstraction to learn more discriminative point cloud features. It takes advantages of efficient learning and high-quality proposals of the 3D voxel CNN and the flexible receptive fields of the PointNet-based networks. Specifically, the proposed framework summarizes the 3D scene with a 3D voxel CNN into a small set of keypoints via a novel voxel set abstraction module to save follow-up computations and also to encode representative scene features. Given the high-quality 3D proposals generated by the voxel CNN, the RoI-grid pooling is proposed to abstract proposal-specific features from the keypoints to the RoI-grid points via keypoint set abstraction with multiple receptive fields. Compared with conventional pooling operations, the RoI-grid feature points encode much richer context information for accurately estimating object confidences and locations. Extensive experiments on both the KITTI dataset and the Waymo Open dataset show that our proposed PV-RCNN surpasses state-of-the-art 3D detection methods with remarkable margins by using only point clouds. Code is available at https://github.com/open-mmlab/OpenPCDet.

arXiv commentAccepted by CVPR 2020. arXiv admin note: substantial text overlap with arXiv:2102.00463

References in corpus (10)

• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Submanifold Sparse Convolutional Networks
• Point-Voxel CNN for Efficient 3D Deep Learning
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
• HDNET: Exploiting HD Maps for 3D Object Detection
• Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection
• StarNet: Targeted Computation for Object Detection in Point Clouds
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud
• Voxel-FPN: multi-scale voxel feature aggregation in 3D object detection from point clouds
• Patch Refinement -- Localized 3D Object Detection

Cited by in corpus (19)

• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Stereo CenterNet based 3D Object Detection for Autonomous Driving
• Radar Voxel Fusion for 3D Object Detection
• VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detection
• CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud
• FG-Net: Fast Large-Scale LiDAR Point Clouds Understanding Network Leveraging Correlated Feature Mining and Geometric-Aware Modelling
• Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level
• LiDAR Spoofing Meets the New-Gen: Capability Improvements, Broken Assumptions, and New Attack Strategies
• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection
• PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation
• STAL3D: Unsupervised Domain Adaptation for 3D Object Detection via Collaborating Self-Training and Adversarial Learning
• Voxel Transformer for 3D Object Detection
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection
• M3DSSD: Monocular 3D Single Stage Object Detector
• Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
• Fast Object Classification and Meaningful Data Representation of Segmented Lidar Instances
• RoIFusion: 3D Object Detection from LiDAR and Vision
• Comparative study of subset selection methods for rapid prototyping of 3D object detection algorithms