Joint Flow: Temporal Flow Fields for Multi Person Tracking

Abstract

In this work we propose an online multi person pose tracking approach which works on two consecutive frames $I_{t-1}$ and $I_t$. The general formulation of our temporal network allows to rely on any multi person pose estimation approach as spatial network. From the spatial network we extract image features and pose features for both frames. These features serve as input for our temporal model that predicts Temporal Flow Fields (TFF). These TFF are vector fields which indicate the direction in which each body joint is going to move from frame $I_{t-1}$ to frame $I_t$. This novel representation allows to formulate a similarity measure of detected joints. These similarities are used as binary potentials in a bipartite graph optimization problem in order to perform tracking of multiple poses. We show that these TFF can be learned by a relative small CNN network whilst achieving state-of-the-art multi person pose tracking results.

Accepted to BMVC

References in corpus (4)

• MOT16: A Benchmark for Multi-Object Tracking
• Stacked Hourglass Networks for Human Pose Estimation
• Towards Accurate Multi-person Pose Estimation in the Wild
• Multi-Person Pose Estimation with Local Joint-to-Person Associations

Cited by in corpus (13)

• Self-supervised Keypoint Correspondences for Multi-Person Pose Estimation and Tracking in Videos
• FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks
• Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation
• Exploiting Offset-guided Network for Pose Estimation and Tracking
• Deep Dual Consecutive Network for Human Pose Estimation
• LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking
• Multi-person Articulated Tracking with Spatial and Temporal Embeddings
• Single-Camera Basketball Tracker through Pose and Semantic Feature Fusion
• Combining detection and tracking for human pose estimation in videos
• Multi-View Matching (MVM): Facilitating Multi-Person 3D Pose Estimation Learning with Action-Frozen People Video
• Pose estimator and tracker using temporal flow maps for limbs
• Iterative Greedy Matching for 3D Human Pose Tracking from Multiple Views
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose