FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks

Abstract

Both accuracy and efficiency are significant for pose estimation and tracking in videos. State-of-the-art performance is dominated by two-stages top-down methods. Despite the leading results, these methods are impractical for real-world applications due to their separated architectures and complicated calculation. This paper addresses the task of articulated multi-person pose estimation and tracking towards real-time speed. An end-to-end multi-task network (MTN) is designed to perform human detection, pose estimation, and person re-identification (Re-ID) tasks simultaneously. To alleviate the performance bottleneck caused by scale variation problem, a paradigm which exploits scale-normalized image and feature pyramids (SIFP) is proposed to boost both performance and speed. Given the results of MTN, we adopt an occlusion-aware Re-ID feature strategy in the pose tracking module, where pose information is utilized to infer the occlusion state to make better use of Re-ID feature. In experiments, we demonstrate that the pose estimation and tracking performance improves steadily utilizing SIFP through different backbones. Using ResNet-18 and ResNet-50 as backbones, the overall pose tracking framework achieves competitive performance with 29.4 FPS and 12.2 FPS, respectively. Additionally, occlusion-aware Re-ID feature decreases the identification switches by 37% in the pose tracking process.

References in corpus (12)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• DeepPose: Human Pose Estimation via Deep Neural Networks
• An Overview of Multi-Task Learning in Deep Neural Networks
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• A Survey on Multi-Task Learning
• Face Detection through Scale-Friendly Deep Convolutional Networks
• Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification
• Joint Flow: Temporal Flow Fields for Multi Person Tracking
• Attention-guided Unified Network for Panoptic Segmentation
• Exploiting Offset-guided Network for Pose Estimation and Tracking

Cited by in corpus (6)

• Deep Dual Consecutive Network for Human Pose Estimation
• Multi-Stage HRNet: Multiple Stage High-Resolution Network for Human Pose Estimation
• Combining detection and tracking for human pose estimation in videos
• Human Following for Wheeled Robot with Monocular Pan-tilt Camera
• High Performance Visual Object Tracking with Unified Convolutional Networks
• 15 Keypoints Is All You Need