3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Abstract

3D object tracking in point clouds is still a challenging problem due to the sparsity of LiDAR points in dynamic environments. In this work, we propose a Siamese voxel-to-BEV tracker, which can significantly improve the tracking performance in sparse 3D point clouds. Specifically, it consists of a Siamese shape-aware feature learning network and a voxel-to-BEV target localization network. The Siamese shape-aware feature learning network can capture 3D shape information of the object to learn the discriminative features of the object so that the potential target from the background in sparse point clouds can be identified. To this end, we first perform template feature embedding to embed the template's feature into the potential target and then generate a dense 3D shape to characterize the shape information of the potential target. For localizing the tracked target, the voxel-to-BEV target localization network regresses the target's 2D center and the $z$-axis center from the dense bird's eye view (BEV) feature map in an anchor-free manner. Concretely, we compress the voxelized point cloud along $z$-axis through max pooling to obtain a dense BEV feature map, where the regression of the 2D center and the $z$-axis center can be performed more effectively. Extensive evaluation on the KITTI and nuScenes datasets shows that our method significantly outperforms the current state-of-the-art methods by a large margin.

Accepted by NeurIPS 2021

References in corpus (13)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• High-Speed Tracking with Kernelized Correlation Filters
• Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
• AFDet: Anchor Free One Stage 3D Object Detection
• Probabilistic 3D Multi-Object Tracking for Autonomous Driving
• The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes
• Ocean: Object-aware Anchor-free Tracking
• SiamMan: Siamese Motion-aware Network for Visual Tracking
• P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds
• SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network
• EagerMOT: 3D Multi-Object Tracking via Sensor Fusion