HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking
arXiv:2009.07736 · doi:10.1007/s11263-020-01375-2
Abstract
Multi-Object Tracking (MOT) has been notoriously difficult to evaluate. Previous metrics overemphasize the importance of either detection or association. To address this, we present a novel MOT evaluation metric, HOTA (Higher Order Tracking Accuracy), which explicitly balances the effect of performing accurate detection, association and localization into a single unified metric for comparing trackers. HOTA decomposes into a family of sub-metrics which are able to evaluate each of five basic error types separately, which enables clear analysis of tracking performance. We evaluate the effectiveness of HOTA on the MOTChallenge benchmark, and show that it is able to capture important aspects of MOT performance not previously taken into account by established metrics. Furthermore, we show HOTA scores better align with human visual evaluation of tracking performance.
arXiv commentPre-print. Accepted for Publication in the International Journal of Computer Vision, 19 August 2020. Code is available at https://github.com/JonathonLuiten/HOTA-metrics
References in corpus (5)
- Simple Online and Realtime Tracking
- Tracking without bells and whistles
- Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification
- Track to Reconstruct and Reconstruct to Track
- Online Multi-Object Tracking Framework with the GMPHD Filter and Occlusion Group Management
Cited by in corpus (44)
- SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos
- DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association
- Solve the Puzzle of Instance Segmentation in Videos: A Weakly Supervised Framework with Spatio-Temporal Collaboration
- A Review of Testing Object-Based Environment Perception for Safe Automated Driving
- Visual Object Tracking in First Person Vision
- Visual Multi-Object Tracking with Re-Identification and Occlusion Handling using Labeled Random Finite Sets
- Development and evaluation of automated localisation and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking
- Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events
- SoccerNet 2022 Challenges Results
- SoccerNet 2023 Challenges Results
- Keypoint Promptable Re-Identification
- Contrastive Learning for Multi-Object Tracking with Transformers
- Lost and Found: Overcoming Detector Failures in Online Multi-Object Tracking
- SurgiTrack: Fine-Grained Multi-Class Multi-Tool Tracking in Surgical Videos
- High-temporal-resolution event-based vehicle detection and tracking
- An Uncertainty-Aware Performance Measure for Multi-Object Tracking
- BioDrone: A Bionic Drone-based Single Object Tracking Benchmark for Robust Vision
- Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey
- Towards Frame Rate Agnostic Multi-Object Tracking
- MinkSORT: A 3D deep feature extractor using sparse convolutions to improve 3D multi-object tracking in greenhouse tomato plants
- Benchmarking pig detection and tracking under diverse and challenging conditions
- Learnable Graph Matching: A Practical Paradigm for Data Association
- SIRA: Scalable Inter-frame Relation and Association for Radar Perception
- AirCode: A Robust Object Encoding Method
- YOLO11-JDE: Fast and Accurate Multi-Object Tracking with Self-Supervised Re-ID
- SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features
- CAMOT: Camera Angle-aware Multi-Object Tracking
- Motion State: A New Benchmark Multiple Object Tracking
- SPAMming Labels: Efficient Annotations for the Trackers of Tomorrow
- Spatialyze: A Geospatial Video Analytics System with Spatial-Aware Optimizations
- Tracking by Detection and Query: An Efficient End-to-End Framework for Multi-Object Tracking
- MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice
- SALINA: Towards Sustainable Live Sonar Analytics in Wild Ecosystems
- Enhancing In-vehicle Multiple Object Tracking Systems with Embeddable Ising Machines
- One Homography is All You Need: IMM-based Joint Homography and Multiple Object State Estimation
- MEX: Memory-efficient Approach to Referring Multi-Object Tracking
- Quantifying Topology In Pancreatic Tubular Networks From Live Imaging 3D Microscopy
- DMSORT: An efficient parallel maritime multi-object tracking architecture for unmanned vessel platforms
- Engineering an Efficient Object Tracker for Non-Linear Motion
- A comparison between single-stage and two-stage 3D tracking algorithms for greenhouse robotics
- Detection-aware multi-object tracking evaluation
- Online 3D Multi-Camera Perception through Robust 2D Tracking and Depth-based Late Aggregation
- mvHOTA: A multi-view higher order tracking accuracy metric to measure spatial and temporal associations in multi-point detection
- Evaluating the Significance of Outdoor Advertising from Driver's Perspective Using Computer Vision