Learning Policies for Adaptive Tracking with Deep Feature Cascades

Abstract

Visual object tracking is a fundamental and time-critical vision task. Recent years have seen many shallow tracking methods based on real-time pixel-based correlation filters, as well as deep methods that have top performance but need a high-end GPU. In this paper, we learn to improve the speed of deep trackers without losing accuracy. Our fundamental insight is to take an adaptive approach, where easy frames are processed with cheap features (such as pixel values), while challenging frames are processed with invariant but expensive deep features. We formulate the adaptive tracking problem as a decision-making process, and learn an agent to decide whether to locate objects with high confidence on an early layer, or continue processing subsequent layers of a network. This significantly reduces the feed-forward cost for easy frames with distinct or slow-moving objects. We train the agent offline in a reinforcement learning fashion, and further demonstrate that learning all deep layers (so as to provide good features for adaptive tracking) can lead to near real-time average tracking speed of 23 fps on a single CPU while achieving state-of-the-art performance. Perhaps most tellingly, our approach provides a 100X speedup for almost 50% of the time, indicating the power of an adaptive approach.

ICCV 2017 Spotlight, with Supplementary Material

References in corpus (2)

• Siamese Instance Search for Tracking
• Deep Reinforcement Learning for Visual Object Tracking in Videos

Cited by in corpus (13)

• A Twofold Siamese Network for Real-Time Object Tracking
• Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
• Benchmarking Deep Trackers on Aerial Videos
• Attention Control with Metric Learning Alignment for Image Set-based Recognition
• Dependency-aware Attention Control for Unconstrained Face Recognition with Image Sets
• A Review of Visual Trackers and Analysis of its Application to Mobile Robot
• TCDCaps: Visual Tracking via Cascaded Dense Capsules
• DensSiam: End-to-End Densely-Siamese Network with Self-Attention Model for Object Tracking
• Multi-shot Pedestrian Re-identification via Sequential Decision Making
• Real-Time Object Tracking via Meta-Learning: Efficient Model Adaptation and One-Shot Channel Pruning
• Towards a Better Match in Siamese Network Based Visual Object Tracker
• Multi-Branch Siamese Networks with Online Selection for Object Tracking
• MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild