TRAT: Tracking by Attention Using Spatio-Temporal Features

Abstract

Robust object tracking requires knowledge of tracked objects' appearance, motion and their evolution over time. Although motion provides distinctive and complementary information especially for fast moving objects, most of the recent tracking architectures primarily focus on the objects' appearance information. In this paper, we propose a two-stream deep neural network tracker that uses both spatial and temporal features. Our architecture is developed over ATOM tracker and contains two backbones: (i) 2D-CNN network to capture appearance features and (ii) 3D-CNN network to capture motion features. The features returned by the two networks are then fused with attention based Feature Aggregation Module (FAM). Since the whole architecture is unified, it can be trained end-to-end. The experimental results show that the proposed tracker TRAT (TRacking by ATtention) achieves state-of-the-art performance on most of the benchmarks and it significantly outperforms the baseline ATOM tracker.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• High-Speed Tracking with Kernelized Correlation Filters
• Two-Stream Convolutional Networks for Action Recognition in Videos
• GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
• Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
• Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network
• Transferring Rich Feature Hierarchies for Robust Visual Tracking
• DCFNet: Discriminant Correlation Filters Network for Visual Tracking
• Multiview Hessian Regularization for Image Annotation
• DeepTrack: Learning Discriminative Feature Representations Online for Robust Visual Tracking
• Good Features to Correlate for Visual Tracking
• You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localization
• SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking