Distilled Siamese Networks for Visual Tracking

Abstract

In recent years, Siamese network based trackers have significantly advanced the state-of-the-art in real-time tracking. Despite their success, Siamese trackers tend to suffer from high memory costs, which restrict their applicability to mobile devices with tight memory budgets. To address this issue, we propose a distilled Siamese tracking framework to learn small, fast and accurate trackers (students), which capture critical knowledge from large Siamese trackers (teachers) by a teacher-students knowledge distillation model. This model is intuitively inspired by the one teacher vs. multiple students learning method typically employed in schools. In particular, our model contains a single teacher-student distillation module and a student-student knowledge sharing mechanism. The former is designed using a tracking-specific distillation strategy to transfer knowledge from a teacher to students. The latter is utilized for mutual learning between students to enable in-depth knowledge understanding. Extensive empirical evaluations on several popular Siamese trackers demonstrate the generality and effectiveness of our framework. Moreover, the results on five tracking benchmarks show that the proposed distilled trackers achieve compression rates of up to 18$\times$ and frame-rates of $265$ FPS, while obtaining comparable tracking accuracy compared to base models.

Accepted by IEEE T-PAMI

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• High-Speed Tracking with Kernelized Correlation Filters
• FitNets: Hints for Thin Deep Nets
• Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
• Learning Spatially Regularized Correlation Filters for Visual Tracking
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Deep Learning with Limited Numerical Precision
• Transferring Rich Feature Hierarchies for Robust Visual Tracking
• N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning
• Real-Time Correlation Tracking via Joint Model Compression and Transfer

Cited by in corpus (8)

• Adaptive Siamese Tracking with a Compact Latent Network
• Deeply Supervised Skin Lesions Diagnosis with Stage and Branch Attention
• Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam Search
• Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering
• OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds
• Weighted Bayesian Gaussian Mixture Model for Roadside LiDAR Object Detection
• Human Pose Transfer with Augmented Disentangled Feature Consistency
• Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds