Deep Learning in Video Multi-Object Tracking: A Survey
arXiv:1907.12740 · doi:10.1016/j.neucom.2019.11.023
Abstract
The problem of Multiple Object Tracking (MOT) consists in following the trajectory of different objects in a sequence, usually a video. In recent years, with the rise of Deep Learning, the algorithms that provide a solution to this problem have benefited from the representational power of deep models. This paper provides a comprehensive survey on works that employ Deep Learning models to solve the task of MOT on single-camera videos. Four main steps in MOT algorithms are identified, and an in-depth review of how Deep Learning was employed in each one of these stages is presented. A complete experimental comparison of the presented works on the three MOTChallenge datasets is also provided, identifying a number of similarities among the top-performing methods and presenting some possible future research directions.
Accepted in Neurocomputing, 2019. New in v4: updated license in compliance with Elsevier policy. Main text: 29 pages, 10 figures, 7 tables. Summary table in appendix at the end of the paper
References in corpus (24)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- YOLOv3: An Incremental Improvement
- High-Speed Tracking with Kernelized Correlation Filters
- Simple Online and Realtime Tracking
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- In Defense of the Triplet Loss for Person Re-Identification
- Wide Residual Networks
- MOT16: A Benchmark for Multi-Object Tracking
- On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
- Multiple Object Tracking: A Literature Review
- MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking
- Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification
- Deeply-Learned Part-Aligned Representations for Person Re-Identification
- UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking
- Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking
- Deep Continuous Conditional Random Fields with Asymmetric Inter-object Constraints for Online Multi-object Tracking
- CVPR19 Tracking and Detection Challenge: How crowded can it get?
- A Hybrid Data Association Framework for Robust Online Multi-Object Tracking
- Memory Based Online Learning of Deep Representations from Video Streams
- Similarity Mapping with Enhanced Siamese Network for Multi-Object Tracking
- Eliminating Exposure Bias and Loss-Evaluation Mismatch in Multiple Object Tracking
- Who did What at Where and When: Simultaneous Multi-Person Tracking and Activity Recognition
- SOT for MOT
Cited by in corpus (50)
- A Survey of Sound Source Localization with Deep Learning Methods
- A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects
- Cell Segmentation and Tracking using CNN-Based Distance Predictions and a Graph-Based Matching Strategy
- Transformers in Single Object Tracking: An Experimental Survey
- Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI
- TAU: A Framework for Video-Based Traffic Analytics Leveraging Artificial Intelligence and Unmanned Aerial Systems
- Physics-Informed Supervised Residual Learning for Electromagnetic Modeling
- DEFT: Detection Embeddings for Tracking
- Adaptive binarization based on fuzzy integrals
- Self-Supervised Small Soccer Player Detection and Tracking
- Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining
- Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
- Optical tracking in team sports
- Regular Polytope Networks
- Simple Unsupervised Multi-Object Tracking
- Global Instance Tracking: Locating Target More Like Humans
- A CRF-based Framework for Tracklet Inactivation in Online Multi-Object Tracking
- Transformers for Multi-Object Tracking on Point Clouds
- A Plausibility-based Fault Detection Method for High-level Fusion Perception Systems
- GCNNMatch: Graph Convolutional Neural Networks for Multi-Object Tracking via Sinkhorn Normalization
- Rank-based verification for long-term face tracking in crowded scenes
- Detection and Tracking Meet Drones Challenge
- SOTVerse: A User-defined Task Space of Single Object Tracking
- Editing Motion Graphics Video via Motion Vectorization and Transformation
- Real-Time Siamese Multiple Object Tracker with Enhanced Proposals
- The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory
- Cascaded Structure Tensor Framework for Robust Identification of Heavily Occluded Baggage Items from Multi-Vendor X-ray Scans
- Multi-object Tracking via End-to-end Tracklet Searching and Ranking
- Learning a Proposal Classifier for Multiple Object Tracking
- YOLO11-JDE: Fast and Accurate Multi-Object Tracking with Self-Supervised Re-ID
- SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features
- How to Track and Segment Fish without Human Annotations: A Self-Supervised Deep Learning Approach
- PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation
- Unmanned Aerial Vehicle Visual Detection and Tracking using Deep Neural Networks: A Performance Benchmark
- Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data
- S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking with Adaptive Spatio-Temporal Appearance Representations
- Multi-Object Tracking with Deep Learning Ensemble for Unmanned Aerial System Applications
- A Deep Learning Bidirectional Temporal Tracking Algorithm for Automated Blood Cell Counting from Non-invasive Capillaroscopy Videos
- A review of UAV Visual Detection and Tracking Methods
- Segmentation with Residual Attention U-Net and an Edge-Enhancement Approach Preserves Cell Shape Features
- CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking
- Advancing biological super-resolution microscopy through deep learning: a brief review
- Towards in-store multi-person tracking using head detection and track heatmaps
- Engineering an Efficient Object Tracker for Non-Linear Motion
- Limitation of Acyclic Oriented Graphs Matching as Cell Tracking Accuracy Measure when Evaluating Mitosis
- TagMe: GPS-Assisted Automatic Object Annotation in Videos
- Appearance-free Tripartite Matching for Multiple Object Tracking
- Psychophysical Evaluation of Deep Re-Identification Models
- Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation
- Clique: Spatiotemporal Object Re-identification at the City Scale