TrTr: Visual Tracking with Transformer
arXiv:2105.03817
Abstract
Template-based discriminative trackers are currently the dominant tracking methods due to their robustness and accuracy, and the Siamese-network-based methods that depend on cross-correlation operation between features extracted from template and search images show the state-of-the-art tracking performance. However, general cross-correlation operation can only obtain relationship between local patches in two feature maps. In this paper, we propose a novel tracker network based on a powerful attention mechanism called Transformer encoder-decoder architecture to gain global and rich contextual interdependencies. In this new architecture, features of the template image is processed by a self-attention module in the encoder part to learn strong context information, which is then sent to the decoder part to compute cross-attention with the search image features processed by another self-attention module. In addition, we design the classification and regression heads using the output of Transformer to localize target based on shape-agnostic anchor. We extensively evaluate our tracker TrTr, on VOT2018, VOT2019, OTB-100, UAV, NfS, TrackingNet, and LaSOT benchmarks and our method performs favorably against state-of-the-art algorithms. Training code and pretrained models are available at https://github.com/tongtybj/TrTr.
11 pages, 5 figures
References in corpus (6)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Sequence to Sequence Learning with Neural Networks
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
- TransTrack: Multiple Object Tracking with Transformer
- End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures