Opening up Open-World Tracking
arXiv:2104.11221
Abstract
Tracking and detecting any object, including ones never-seen-before during model training, is a crucial but elusive capability of autonomous systems. An autonomous agent that is blind to never-seen-before objects poses a safety hazard when operating in the real world - and yet this is how almost all current systems work. One of the main obstacles towards advancing tracking any object is that this task is notoriously difficult to evaluate. A benchmark that would allow us to perform an apples-to-apples comparison of existing efforts is a crucial first step towards advancing this important research field. This paper addresses this evaluation deficit and lays out the landscape and evaluation methodology for detecting and tracking both known and unknown objects in the open-world setting. We propose a new benchmark, TAO-OW: Tracking Any Object in an Open World, analyze existing efforts in multi-object tracking, and construct a baseline for this task while highlighting future challenges. We hope to open a new front in multi-object tracking research that will hopefully bring us a step closer to intelligent systems that can operate safely in the real world. https://openworldtracking.github.io/
arXiv commentCVPR 2022 (Oral). https://openworldtracking.github.io/
References in corpus (18)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Distributed Representations of Words and Phrases and their Compositionality
- Prototypical Networks for Few-shot Learning
- Simple Online and Realtime Tracking
- Zero-Shot Learning by Convex Combination of Semantic Embeddings
- Learning to Segment Object Candidates
- A Novel Performance Evaluation Methodology for Single-Target Trackers
- Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
- Zero Shot Recognition with Unreliable Attributes
- STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos
- Track to Reconstruct and Reconstruct to Track
- The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation
- Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
- SCALOR: Generative World Models with Scalable Object Representations
- A Detailed Rubric for Motion Segmentation
- Identifying Unknown Instances for Autonomous Driving
- 1st Place Solution to ECCV-TAO-2020: Detect and Represent Any Object for Tracking
- STEP: Segmenting and Tracking Every Pixel