TransTrack: Multiple Object Tracking with Transformer
arXiv:2012.15460
Abstract
In this work, we propose TransTrack, a simple but efficient scheme to solve the multiple object tracking problems. TransTrack leverages the transformer architecture, which is an attention-based query-key mechanism. It applies object features from the previous frame as a query of the current frame and introduces a set of learned object queries to enable detecting new-coming objects. It builds up a novel joint-detection-and-tracking paradigm by accomplishing object detection and object association in a single shot, simplifying complicated multi-step settings in tracking-by-detection methods. On MOT17 and MOT20 benchmark, TransTrack achieves 74.5\% and 64.5\% MOTA, respectively, competitive to the state-of-the-art methods. We expect TransTrack to provide a novel perspective for multiple object tracking. The code is available at: \url{https://github.com/PeizeSun/TransTrack}.
arXiv commentupdate MOT17 and MOT20
References in corpus (11)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Decoupled Weight Decay Regularization
- Simple Online and Realtime Tracking
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- MOT16: A Benchmark for Multi-Object Tracking
- T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
- MOT20: A benchmark for multi object tracking in crowded scenes
- CrowdHuman: A Benchmark for Detecting Human in a Crowd
- Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
- End-to-End Video Instance Segmentation with Transformers
Cited by in corpus (32)
- A Survey on Visual Transformer
- SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- LocalViT: Analyzing Locality in Vision Transformers
- TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers
- A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions
- ByteTrack: Multi-Object Tracking by Associating Every Detection Box
- TrTr: Visual Tracking with Transformer
- Real-time 3D Single Object Tracking with Transformer
- Shadow-Background-Noise 3D Spatial Decomposition Using Sparse Low-Rank Gaussian Properties for Video-SAR Moving Target Shadow Enhancement
- 3D Object Tracking with Transformer
- A Video Is Worth Three Views: Trigeminal Transformers for Video-based Person Re-identification
- BACTrack: Building Appearance Collection for Aerial Tracking
- Segmenting Transparent Object in the Wild with Transformer
- Contrastive Learning for Multi-Object Tracking with Transformers
- Learning Data Association for Multi-Object Tracking using Only Coordinates
- Salient Mask-Guided Vision Transformer for Fine-Grained Classification
- SurgiTrack: Fine-Grained Multi-Class Multi-Tool Tracking in Surgical Videos
- MOTR: End-to-End Multiple-Object Tracking with Transformer
- CoSformer: Detecting Co-Salient Object with Transformers
- Track to Detect and Segment: An Online Multi-Object Tracker
- Learnable Graph Matching: A Practical Paradigm for Data Association
- CAMOT: Camera Angle-aware Multi-Object Tracking
- End-to-End Video Object Detection with Spatial-Temporal Transformers
- CATs: Cost Aggregation Transformers for Visual Correspondence
- S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking with Adaptive Spatio-Temporal Appearance Representations
- Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking
- UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model
- Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers
- A Survey of Fish Tracking Techniques Based on Computer Vision
- Attend to Who You Are: Supervising Self-Attention for Keypoint Detection and Instance-Aware Association
- All You Can Embed: Natural Language based Vehicle Retrieval with Spatio-Temporal Transformers
- Multi-Object Tracking with Hallucinated and Unlabeled Videos