Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies

Abstract

The majority of existing solutions to the Multi-Target Tracking (MTT) problem do not combine cues in a coherent end-to-end fashion over a long period of time. However, we present an online method that encodes long-term temporal dependencies across multiple cues. One key challenge of tracking methods is to accurately track occluded targets or those which share similar appearance properties with surrounding objects. To address this challenge, we present a structure of Recurrent Neural Networks (RNN) that jointly reasons on multiple cues over a temporal window. We are able to correct many data association errors and recover observations from an occluded state. We demonstrate the robustness of our data-driven approach by tracking multiple targets using their appearance, motion, and even interactions. Our method outperforms previous works on multiple publicly available datasets including the challenging MOT benchmark.

References in corpus (6)

• Social Force Model for Pedestrian Dynamics
• MOT16: A Benchmark for Multi-Object Tracking
• MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
• Temporal Dynamic Appearance Modeling for Online Multi-Person Tracking
• A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects
• Deep Learning of Appearance Models for Online Object Tracking

Cited by in corpus (9)

• Tracklet Association Tracker: An End-to-End Learning-based Association Approach for Multi-Object Tracking
• Graph Neural Based End-to-end Data Association Framework for Online Multiple-Object Tracking
• FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking
• Exploit the Connectivity: Multi-Object Tracking with TrackletNet
• Multiple People Tracking Using Hierarchical Deep Tracklet Re-identification
• CAR-Net: Clairvoyant Attentive Recurrent Network
• Towards a Principled Integration of Multi-Camera Re-Identification and Tracking through Optimal Bayes Filters
• Online Visual Robot Tracking and Identification using Deep LSTM Networks
• Active Learning for Structured Prediction from Partially Labeled Data