Center-based 3D Object Detection and Tracking

Abstract

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-the-art performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, CenterPoint outperforms all previous single model method by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

update nuScenes and Waymo results

References in corpus (11)

• Improving neural networks by preventing co-adaptation of feature detectors
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
• StarNet: Targeted Computation for Object Detection in Point Clouds
• AFDet: Anchor Free One Stage 3D Object Detection
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud
• Probabilistic 3D Multi-Object Tracking for Autonomous Driving
• 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
• GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving
• Identifying Unknown Instances for Autonomous Driving
• Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection
• An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds

Cited by in corpus (20)

• One Million Scenes for Autonomous Driving: ONCE Dataset
• Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting
• Radar Voxel Fusion for 3D Object Detection
• Auto4D: Learning to Label 4D Objects from Sequential Point Clouds
• Exploring Data Augmentation for Multi-Modality 3D Object Detection
• Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots
• NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
• Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection
• FlowMOT: 3D Multi-Object Tracking by Scene Flow Association
• PolarStream: Streaming Lidar Object Detection and Segmentation with Polar Pillars
• Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection
• VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
• EagerMOT: 3D Multi-Object Tracking via Sensor Fusion
• CenterAtt: Fast 2-stage Center Attention Network
• A two-stage data association approach for 3D Multi-object Tracking
• A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulation
• Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation
• Tracking from Patterns: Learning Corresponding Patterns in Point Clouds for 3D Object Tracking
• R-AGNO-RPN: A LIDAR-Camera Region Deep Network for Resolution-Agnostic Detection