3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

Abstract

3D single object tracking is a key issue for autonomous following robot, where the robot should robustly track and accurately localize the target for efficient following. In this paper, we propose a 3D tracking method called 3D-SiamRPN Network to track a single target object by using raw 3D point cloud data. The proposed network consists of two subnetworks. The first subnetwork is feature embedding subnetwork which is used for point cloud feature extraction and fusion. In this subnetwork, we first use PointNet++ to extract features of point cloud from template and search branches. Then, to fuse the information of features in the two branches and obtain their similarity, we propose two cross correlation modules, named Pointcloud-wise and Point-wise respectively. The second subnetwork is region proposal network(RPN), which is used to get the final 3D bounding box of the target object based on the fusion feature from cross correlation modules. In this subnetwork, we utilize the regression and classification branches of a region proposal subnetwork to obtain proposals and scores, thus get the final 3D bounding box of the target object. Experimental results on KITTI dataset show that our method has a competitive performance in both Success and Precision compared to the state-of-the-art methods, and could run in real-time at 20.8 FPS. Additionally, experimental results on H3D dataset demonstrate that our method also has good generalization ability and could achieve good tracking performance in a new scene without re-training.

References in corpus (8)

• High-Speed Tracking with Kernelized Correlation Filters
• Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
• A Novel Performance Evaluation Methodology for Single-Target Trackers
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• End-to-end Learning of Multi-sensor 3D Tracking by Detection
• Efficient Bird Eye View Proposals for 3D Siamese Tracking
• 3D Point Cloud Descriptors in Hand-crafted and Deep Learning Age: State-of-the-Art
• PointIT: A Fast Tracking Framework Based on 3D Instance Segmentation

Cited by in corpus (6)

• Real-time 3D Single Object Tracking with Transformer
• P2P: Part-to-Part Motion Cues Guide a Strong Tracking Framework for LiDAR Point Clouds
• Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wild
• VPIT: Real-time Embedded Single Object 3D Tracking Using Voxel Pseudo Images
• Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds
• Simulation to Reality: Testbeds and Architectures for Connected and Automated Vehicles