15 Keypoints Is All You Need
arXiv:1912.02323
Abstract
Pose tracking is an important problem that requires identifying unique human pose-instances and matching them temporally across different frames of a video. However, existing pose tracking methods are unable to accurately model temporal relationships and require significant computation, often computing the tracks offline. We present an efficient Multi-person Pose Tracking method, KeyTrack, that only relies on keypoint information without using any RGB or optical flow information to track human keypoints in real-time. Keypoints are tracked using our Pose Entailment method, in which, first, a pair of pose estimates is sampled from different frames in a video and tokenized. Then, a Transformer-based network makes a binary classification as to whether one pose temporally follows another. Furthermore, we improve our top-down pose estimation method with a novel, parameter-free, keypoint refinement technique that improves the keypoint estimates used during the Pose Entailment step. We achieve state-of-the-art results on the PoseTrack'17 and the PoseTrack'18 benchmarks while using only a fraction of the computation required by most other methods for computing the tracking information.
CVPR 2020
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- DeepPose: Human Pose Estimation via Deep Neural Networks
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- MMDetection: Open MMLab Detection Toolbox and Benchmark
- A large annotated corpus for learning natural language inference
- Learning Video Representations using Contrastive Bidirectional Transformer
- Stand-Alone Self-Attention in Vision Models
- End-to-End Dense Video Captioning with Masked Transformer
- FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks
- Spatial-Temporal Relation Networks for Multi-Object Tracking
- Learning Temporal Pose Estimation from Sparsely-Labeled Videos
- LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking