Spatial-Temporal Relation Networks for Multi-Object Tracking

Abstract

Recent progress in multiple object tracking (MOT) has shown that a robust similarity score is key to the success of trackers. A good similarity score is expected to reflect multiple cues, e.g. appearance, location, and topology, over a long period of time. However, these cues are heterogeneous, making them hard to be combined in a unified network. As a result, existing methods usually encode them in separate networks or require a complex training approach. In this paper, we present a unified framework for similarity measurement which could simultaneously encode various cues and perform reasoning across both spatial and temporal domains. We also study the feature representation of a tracklet-object pair in depth, showing a proper design of the pair features can well empower the trackers. The resulting approach is named spatial-temporal relation networks (STRN). It runs in a feed-forward way and can be trained in an end-to-end manner. The state-of-the-art accuracy was achieved on all of the MOT15-17 benchmarks using public detection and online settings.

References in corpus (8)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MOT16: A Benchmark for Multi-Object Tracking
• MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
• Interaction Networks for Learning about Objects, Relations and Physics
• A simple neural network module for relational reasoning
• Non-local Neural Networks
• Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking
• A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects

Cited by in corpus (3)

• Learning a Proposal Classifier for Multiple Object Tracking
• ArTIST: Autoregressive Trajectory Inpainting and Scoring for Tracking
• 15 Keypoints Is All You Need