End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning

Abstract

We study active object tracking, where a tracker takes visual observations (i.e., frame sequences) as input and produces the corresponding camera control signals as output (e.g., move forward, turn left, etc.). Conventional methods tackle tracking and camera control tasks separately, and the resulting system is difficult to tune jointly. These methods also require significant human efforts for image labeling and expensive trial-and-error system tuning in the real world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning. A ConvNet-LSTM function approximator is adopted for the direct frame-to-action prediction. We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training. The tracker trained in simulators (ViZDoom and Unreal Engine) demonstrates good generalization behaviors in the case of unseen object moving paths, unseen object appearances, unseen backgrounds, and distracting objects. The system is robust and can restore tracking after occasional lost of the target being tracked. We also find that the tracking ability, obtained solely from simulators, can potentially transfer to real-world scenarios. We demonstrate successful examples of such transfer, via experiments over the VOT dataset and the deployment of a real-world robot using the proposed active tracker trained in simulation.

arXiv commentTo appear in Transactions on Pattern Analysis and Machine Intelligence. arXiv admin note: text overlap with arXiv:1705.10561

References in corpus (9)

• YOLOv3: An Incremental Improvement
• High-Speed Tracking with Kernelized Correlation Filters
• Multiple Object Tracking: A Literature Review
• A Novel Performance Evaluation Methodology for Single-Target Trackers
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
• Tree-Structured Reinforcement Learning for Sequential Object Localization
• Sim2Real View Invariant Visual Servoing by Recurrent Control
• Real-time visual tracking by deep reinforced decision making

Cited by in corpus (3)

• Unsupervised Deep Tracking
• AdapNet: Adaptability Decomposing Encoder-Decoder Network for Weakly Supervised Action Recognition and Localization
• Autonomous Curiosity for Real-Time Training Onboard Robotic Agents