Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking
arXiv:2103.11681
Abstract
In video object tracking, there exist rich temporal contexts among successive frames, which have been largely overlooked in existing trackers. In this work, we bridge the individual video frames and explore the temporal contexts across them via a transformer architecture for robust object tracking. Different from classic usage of the transformer in natural language processing tasks, we separate its encoder and decoder into two parallel branches and carefully design them within the Siamese-like tracking pipelines. The transformer encoder promotes the target templates via attention-based feature reinforcement, which benefits the high-quality tracking model generation. The transformer decoder propagates the tracking cues from previous templates to the current frame, which facilitates the object searching process. Our transformer-assisted tracking framework is neat and trained in an end-to-end manner. With the proposed transformer, a simple Siamese matching approach is able to outperform the current top-performing trackers. By combining our transformer with the recent discriminative tracking pipeline, our method sets several new state-of-the-art records on prevalent tracking benchmarks.
To appear in CVPR 2021 (Oral)
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Distilling the Knowledge in a Neural Network
- A Simple Framework for Contrastive Learning of Visual Representations
- GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
- Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
- End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
- Siamese Instance Search for Tracking
- Deeper and Wider Siamese Networks for Real-Time Visual Tracking
- Distractor-aware Siamese Networks for Visual Object Tracking
- Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking
- CREST: Convolutional Residual Learning for Visual Tracking
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild
- Need for Speed: A Benchmark for Higher Frame Rate Object Tracking
- Unsupervised Deep Tracking
- Target-Aware Deep Tracking
- SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking
- Learning the Model Update for Siamese Trackers