Joint Spatial-Temporal Optimization for Stereo 3D Object Tracking

Abstract

Directly learning multiple 3D objects motion from sequential images is difficult, while the geometric bundle adjustment lacks the ability to localize the invisible object centroid. To benefit from both the powerful object understanding skill from deep neural network meanwhile tackle precise geometry modeling for consistent trajectory estimation, we propose a joint spatial-temporal optimization-based stereo 3D object tracking method. From the network, we detect corresponding 2D bounding boxes on adjacent images and regress an initial 3D bounding box. Dense object cues (local depth and local coordinates) that associating to the object centroid are then predicted using a region-based network. Considering both the instant localization accuracy and motion consistency, our optimization models the relations between the object centroid and observed cues into a joint spatial-temporal error function. All historic cues will be summarized to contribute to the current estimation by a per-frame marginalization strategy without repeated computation. Quantitative evaluation on the KITTI tracking dataset shows our approach outperforms previous image-based 3D tracking methods by significant margins. We also report extensive results on multiple categories and larger datasets (KITTI raw and Argoverse Tracking) for future benchmarking.

cvpr2020

References in corpus (14)

• VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
• Deep Residual Learning for Image Recognition
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Feature Pyramid Networks for Object Detection
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• Joint 3D Proposal Generation and Object Detection from View Aggregation
• Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
• Multi-View 3D Object Detection Network for Autonomous Driving
• Joint Monocular 3D Vehicle Detection and Tracking
• 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
• Multi-Task Multi-Sensor Fusion for 3D Object Detection