SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks

Abstract

Siamese network based trackers formulate tracking as convolutional feature cross-correlation between target template and searching region. However, Siamese trackers still have accuracy gap compared with state-of-the-art algorithms and they cannot take advantage of feature from deep networks, such as ResNet-50 or deeper. In this work we prove the core reason comes from the lack of strict translation invariance. By comprehensive theoretical analysis and experimental validations, we break this restriction through a simple yet effective spatial aware sampling strategy and successfully train a ResNet-driven Siamese tracker with significant performance gain. Moreover, we propose a new model architecture to perform depth-wise and layer-wise aggregations, which not only further improves the accuracy but also reduces the model size. We conduct extensive ablation studies to demonstrate the effectiveness of the proposed tracker, which obtains currently the best results on four large tracking benchmarks, including OTB2015, VOT2018, UAV123, and LaSOT. Our model will be released to facilitate further studies based on this problem.

arXiv comment9 pages

References in corpus (4)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
• DCFNet: Discriminant Correlation Filters Network for Visual Tracking
• An In-Depth Analysis of Visual Tracking with Siamese Neural Networks

Cited by in corpus (15)

• Deep Learning for Visual Tracking: A Comprehensive Survey
• End-to-end Learning of Object Motion Estimation from Retinal Events for Event-based Object Tracking
• Unsupervised Cross-Modal Distillation for Thermal Infrared Tracking
• Efficient Bird Eye View Proposals for 3D Siamese Tracking
• How To Train Your Deep Multi-Object Tracker
• Benchmarking Deep Trackers on Aerial Videos
• Learning the Model Update for Siamese Trackers
• Prediction-Tracking-Segmentation
• Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation
• SAMOT: Switcher-Aware Multi-Object Tracking and Still Another MOT Measure
• Pose-Assisted Multi-Camera Collaboration for Active Object Tracking
• Accurate Anchor Free Tracking
• Future Video Synthesis with Object Motion Prediction
• Motion Prediction in Visual Object Tracking
• Multi-Modal Fusion for End-to-End RGB-T Tracking