Distilled Siamese Networks for Visual Tracking

Abstract

In recent years, Siamese network based trackers have significantly advanced the state-of-the-art in real-time tracking. Despite their success, Siamese trackers tend to suffer from high memory costs, which restrict their applicability to mobile devices with tight memory budgets. To address this issue, we propose a distilled Siamese tracking framework to learn small, fast and accurate trackers (students), which capture critical knowledge from large Siamese trackers (teachers) by a teacher-students knowledge distillation model. This model is intuitively inspired by the one teacher vs. multiple students learning method typically employed in schools. In particular, our model contains a single teacher-student distillation module and a student-student knowledge sharing mechanism. The former is designed using a tracking-specific distillation strategy to transfer knowledge from a teacher to students. The latter is utilized for mutual learning between students to enable in-depth knowledge understanding. Extensive empirical evaluations on several popular Siamese trackers demonstrate the generality and effectiveness of our framework. Moreover, the results on five tracking benchmarks show that the proposed distilled trackers achieve compression rates of up to 18$\times$ and frame-rates of $265$ FPS, while obtaining comparable tracking accuracy compared to base models.

arXiv commentAccepted by IEEE T-PAMI

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• High-Speed Tracking with Kernelized Correlation Filters
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• FitNets: Hints for Thin Deep Nets
• Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
• Learning Spatially Regularized Correlation Filters for Visual Tracking
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Do Deep Nets Really Need to be Deep?
• Deep Learning with Limited Numerical Precision
• Born Again Neural Networks
• Transferring Rich Feature Hierarchies for Robust Visual Tracking
• Context-aware Deep Feature Compression for High-speed Visual Tracking
• Learning feed-forward one-shot learners
• Quadruplet Network with One-Shot Learning for Fast Visual Object Tracking
• Unifying distillation and privileged information
• N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning
• Real-Time Correlation Tracking via Joint Model Compression and Transfer
• Object Tracking by Reconstruction with View-Specific Discriminative Correlation Filters

Cited by in corpus (8)

• Adaptive Siamese Tracking with a Compact Latent Network
• Deeply Supervised Skin Lesions Diagnosis with Stage and Branch Attention
• Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam Search
• Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering
• OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds
• Weighted Bayesian Gaussian Mixture Model for Roadside LiDAR Object Detection
• Human Pose Transfer with Augmented Disentangled Feature Consistency
• Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds