3D Multi-Object Tracking: A Baseline and New Evaluation Metrics

Abstract

3D multi-object tracking (MOT) is an essential component for many applications such as autonomous driving and assistive robotics. Recent work on 3D MOT focuses on developing accurate systems giving less attention to practical considerations such as computational cost and system complexity. In contrast, this work proposes a simple real-time 3D MOT system. Our system first obtains 3D detections from a LiDAR point cloud. Then, a straightforward combination of a 3D Kalman filter and the Hungarian algorithm is used for state estimation and data association. Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in the 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods. Therefore, we propose a new 3D MOT evaluation tool along with three new metrics to comprehensively evaluate 3D MOT methods. We show that, although our system employs a combination of classical MOT modules, we achieve state-of-the-art 3D MOT performance on two 3D MOT benchmarks (KITTI and nuScenes). Surprisingly, although our system does not use any 2D data as inputs, we achieve competitive performance on the KITTI 2D MOT leaderboard. Our proposed system runs at a rate of $207.4$ FPS on the KITTI dataset, achieving the fastest speed among all modern MOT systems. To encourage standardized 3D MOT evaluation, our system and evaluation code are made publicly available at https://github.com/xinshuoweng/AB3DMOT.

Accepted at IROS 2020

References in corpus (11)

• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• HDNET: Exploiting HD Maps for 3D Object Detection
• Simple Online and Realtime Tracking with a Deep Association Metric
• Learning Shape Representations for Clothing Variations in Person Re-Identification
• How To Train Your Deep Multi-Object Tracker
• PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
• Visual Compiler: Synthesizing a Scene-Specific Pedestrian Detector and Pose Estimator
• When We First Met: Visual-Inertial Person Localization for Co-Robot Rendezvous
• Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking

Cited by in corpus (15)

• Scalability in Perception for Autonomous Driving: Waymo Open Dataset
• Center-based 3D Object Detection and Tracking
• DEFT: Detection Embeddings for Tracking
• Probabilistic 3D Multi-Object Tracking for Autonomous Driving
• SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking
• TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking
• Tracking Objects as Points
• GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning
• PV-RCNN: The Top-Performing LiDAR-only Solutions for 3D Detection / 3D Tracking / Domain Adaptation of Waymo Open Dataset Challenges
• CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking
• PC-DAN: Point Cloud based Deep Affinity Network for 3D Multi-Object Tracking (Accepted as an extended abstract in JRDB-ACT Workshop at CVPR21)
• Online Multi-Target Tracking for Maneuvering Vehicles in Dynamic Road Context
• Joint Spatial-Temporal Optimization for Stereo 3D Object Tracking
• Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation
• Vehicular Multi-object Tracking with Persistent Detector Failures