Multi-Context Attention for Human Pose Estimation

Abstract

In this paper, we propose to incorporate convolutional neural networks with a multi-context attention mechanism into an end-to-end framework for human pose estimation. We adopt stacked hourglass networks to generate attention maps from features at multiple resolutions with various semantics. The Conditional Random Field (CRF) is utilized to model the correlations among neighboring regions in the attention map. We further combine the holistic attention model, which focuses on the global consistency of the full human body, and the body part attention model, which focuses on the detailed description for different body parts. Hence our model has the ability to focus on different granularity from local salient regions to global semantic-consistent spaces. Additionally, we design novel Hourglass Residual Units (HRUs) to increase the receptive field of the network. These units are extensions of residual units with a side branch incorporating filters with larger receptive fields, hence features with various scales are learned and combined within the HRUs. The effectiveness of the proposed multi-context attention mechanism and the hourglass residual units is evaluated on two widely used human pose estimation benchmarks. Our approach outperforms all existing methods on both benchmarks over all the body parts.

The first two authors contribute equally to this work

Cited by in corpus (33)

• Attention Mechanisms in Computer Vision: A Survey
• Human Pose Regression by Combining Indirect Part Detection and Contextual Information
• Numerical Coordinate Regression with Convolutional Neural Networks
• Simple Baselines for Human Pose Estimation and Tracking
• Rethinking on Multi-Stage Networks for Human Pose Estimation
• Attention-Aware Compositional Network for Person Re-identification
• Learning Feature Pyramids for Human Pose Estimation
• TFPose: Direct Human Pose Estimation with Transformers
• Self Adversarial Training for Human Pose Estimation
• 3D Human Pose Estimation in the Wild by Adversarial Learning
• Look at Boundary: A Boundary-Aware Face Alignment Algorithm
• Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition
• Multi-Scale Structure-Aware Network for Human Pose Estimation
• In-bed Pressure-based Pose Estimation using Image Space Representation Learning
• Pose2Seg: Detection Free Human Instance Segmentation
• TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
• RePose: Learning Deep Kinematic Priors for Fast Human Pose Estimation
• Dense Recurrent Neural Networks for Scene Labeling
• 3D Human Pose Estimation with 2D Marginal Heatmaps
• CoKe: Localized Contrastive Learning for Robust Keypoint Detection
• Human Motion Analysis with Deep Metric Learning
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
• Multi-scale Aggregation R-CNN for 2D Multi-person Pose Estimation
• Towards Highly Accurate and Stable Face Alignment for High-Resolution Videos
• See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks
• Deep Imbalanced Attribute Classification using Visual Attention Aggregation
• Efficient Human Pose Estimation by Learning Deeply Aggregated Representations
• Computer Vision and Abnormal Patient Gait Assessment a Comparison of Machine Learning Models
• Skeleton Transformer Networks: 3D Human Pose and Skinned Mesh from Single RGB Image
• What I See Is What You See: Joint Attention Learning for First and Third Person Video Co-analysis
• Scene Parsing via Dense Recurrent Neural Networks with Attentional Selection
• On Improving the Generalization of Face Recognition in the Presence of Occlusions
• Joint Voxel and Coordinate Regression for Accurate 3D Facial Landmark Localization