TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model
arXiv:2006.05683
Abstract
Multi-object tracking is a fundamental vision problem that has been studied for a long time. As deep learning brings excellent performances to object detection algorithms, Tracking by Detection (TBD) has become the mainstream tracking framework. Despite the success of TBD, this two-step method is too complicated to train in an end-to-end manner and induces many challenges as well, such as insufficient exploration of video spatial-temporal information, vulnerability when facing object occlusion, and excessive reliance on detection results. To address these challenges, we propose a concise end-to-end model TubeTK which only needs one step training by introducing the ``bounding-tube" to indicate temporal-spatial locations of objects in a short video clip. TubeTK provides a novel direction of multi-object tracking, and we demonstrate its potential to solve the above challenges without bells and whistles. We analyze the performance of TubeTK on several MOT benchmarks and provide empirical evidence to show that TubeTK has the ability to overcome occlusions to some extent without any ancillary technologies like Re-ID. Compared with other methods that adopt private detection results, our one-stage end-to-end model achieves state-of-the-art performances even if it adopts no ready-made detection results. We hope that the proposed TubeTK model can serve as a simple but strong alternative for video-based MOT task. The code and models are available at https://github.com/BoPang1996/TubeTK.
arXiv commentCVPR-2020 oral paper
References in corpus (20)
- Simple Online and Realtime Tracking
- MOT16: A Benchmark for Multi-Object Tracking
- Focal Loss for Dense Object Detection
- FCOS: Fully Convolutional One-Stage Object Detection
- MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
- T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
- Decomposing Motion and Content for Natural Video Sequence Prediction
- CenterNet: Keypoint Triplets for Object Detection
- SlowFast Networks for Video Recognition
- Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
- Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification
- Integrated Object Detection and Tracking with Tracklet-Conditioned Detection
- HAKE: Human Activity Knowledge Engine
- Temporal Relational Reasoning in Videos
- Towards Real-Time Multi-Object Tracking
- FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking
- Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution
- Recurrent Autoregressive Networks for Online Multi-Object Tracking
- Beyond Holistic Object Recognition: Enriching Image Understanding with Part States
- SRDA: Generating Instance Segmentation Annotation Via Scanning, Reasoning And Domain Adaptation