Multi-Scale Structure-Aware Network for Human Pose Estimation

Abstract

We develop a robust multi-scale structure-aware neural network for human pose estimation. This method improves the recent deep conv-deconv hourglass models with four key improvements: (1) multi-scale supervision to strengthen contextual feature learning in matching body keypoints by combining feature heatmaps across scales, (2) multi-scale regression network at the end to globally optimize the structural matching of the multi-scale features, (3) structure-aware loss used in the intermediate supervision and at the regression to improve the matching of keypoints and respective neighbors to infer a higher-order matching configurations, and (4) a keypoint masking training scheme that can effectively fine-tune our network to robustly localize occluded keypoints via adjacent matches. Our method can effectively improve state-of-the-art pose estimation methods that suffer from difficulties in scale varieties, occlusions, and complex multi-person scenarios. This multi-scale supervision tightly integrates with the regression network to effectively (i) localize keypoints using the ensemble of multi-scale features, and (ii) infer global pose configuration by maximizing structural consistencies across multiple keypoints and scales. The keypoint masking training enhances these advantages to focus learning on hard occlusion samples. Our method achieves the leading position in the MPII challenge leaderboard among the state-of-the-art methods.

arXiv commentAccepted by ECCV2018

References in corpus (9)

• DeepPose: Human Pose Estimation via Deep Neural Networks
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Human pose estimation via Convolutional Part Heatmap Regression
• Stacked Hourglass Networks for Human Pose Estimation
• Multi-Context Attention for Human Pose Estimation
• Flowing ConvNets for Human Pose Estimation in Videos
• Learning Feature Pyramids for Human Pose Estimation
• Self Adversarial Training for Human Pose Estimation
• Dual Path Networks for Multi-Person Human Pose Estimation

Cited by in corpus (7)

• Rethinking on Multi-Stage Networks for Human Pose Estimation
• How Robust is 3D Human Pose Estimation to Occlusion?
• Improved training of binary networks for human pose estimation and image recognition
• Learning to Estimate 3D Human Pose from Point Cloud
• In-bed Pressure-based Pose Estimation using Image Space Representation Learning
• A Context-and-Spatial Aware Network for Multi-Person Pose Estimation
• Matrix and tensor decompositions for training binary neural networks