Self-supervised Keypoint Correspondences for Multi-Person Pose Estimation and Tracking in Videos

Abstract

Video annotation is expensive and time consuming. Consequently, datasets for multi-person pose estimation and tracking are less diverse and have more sparse annotations compared to large scale image datasets for human pose estimation. This makes it challenging to learn deep learning based models for associating keypoints across frames that are robust to nuisance factors such as motion blur and occlusions for the task of multi-person pose tracking. To address this issue, we propose an approach that relies on keypoint correspondences for associating persons in videos. Instead of training the network for estimating keypoint correspondences on video data, it is trained on a large scale image datasets for human pose estimation using self-supervision. Combined with a top-down framework for human pose estimation, we use keypoints correspondences to (i) recover missed pose detections (ii) associate pose detections across video frames. Our approach achieves state-of-the-art results for multi-frame pose estimation and multi-person pose tracking on the PosTrack $2017$ and PoseTrack $2018$ data sets.

Accepted to ECCV 2020

References in corpus (12)

• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Universal Correspondence Network
• AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
• Rethinking on Multi-Stage Networks for Human Pose Estimation
• Joint-task Self-supervised Learning for Temporal Correspondence
• Multi-Domain Pose Network for Multi-Person Pose Estimation and Tracking
• Joint Flow: Temporal Flow Fields for Multi Person Tracking
• Pose Flow: Efficient Online Pose Tracking
• Exploiting Offset-guided Network for Pose Estimation and Tracking
• LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking
• Generative Partition Networks for Multi-Person Pose Estimation
• Multi-scale Aggregation R-CNN for 2D Multi-person Pose Estimation

Cited by in corpus (6)

• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• Deep Learning-Based Human Pose Estimation: A Survey
• Dual networks based 3D Multi-Person Pose Estimation from Monocular Video
• Lost and Found: Overcoming Detector Failures in Online Multi-Object Tracking
• Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videos
• PoseTrackReID: Dataset Description