From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network

Abstract

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-$A^2$ net). The whole framework consists of the part-aware stage and the part-aggregation stage. Firstly, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-$A^2$ net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data. Code is available at https://github.com/sshaoshuai/PointCloudDet3D.

Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence 2020, code is available at https://github.com/sshaoshuai/PointCloudDet3D

References in corpus (6)

• Semantic Instance Segmentation with a Discriminative Loss Function
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Submanifold Sparse Convolutional Networks
• HDNET: Exploiting HD Maps for 3D Object Detection
• Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection

Cited by in corpus (21)

• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• 3DSSD: Point-based 3D Single Stage Object Detector
• Deep Learning for 3D Point Clouds: A Survey
• LiDAR Spoofing Meets the New-Gen: Capability Improvements, Broken Assumptions, and New Attack Strategies
• Reconfigurable Voxels: A New Representation for LiDAR-Based Point Clouds
• SIENet: Spatial Information Enhancement Network for 3D Object Detection from Point Cloud
• Geometry Uncertainty Projection Network for Monocular 3D Object Detection
• Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection
• OCM3D: Object-Centric Monocular 3D Object Detection
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection
• MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps
• Lidar Point Cloud Guided Monocular 3D Object Detection
• MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views
• Semantic Scene Completion via Integrating Instances and Scene in-the-Loop
• 3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction
• 3D-MAN: 3D Multi-frame Attention Network for Object Detection
• ZoomNet: Part-Aware Adaptive Zooming Neural Network for 3D Object Detection
• SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection
• MBDF-Net: Multi-Branch Deep Fusion Network for 3D Object Detection
• LiDAR R-CNN: An Efficient and Universal 3D Object Detector