Human pose estimation via Convolutional Part Heatmap Regression

Abstract

This paper is on human pose estimation using Convolutional Neural Networks. Our main contribution is a CNN cascaded architecture specifically designed for learning part relationships and spatial context, and robustly inferring pose even for the case of severe part occlusions. To this end, we propose a detection-followed-by-regression CNN cascade. The first part of our cascade outputs part detection heatmaps and the second part performs regression on these heatmaps. The benefits of the proposed architecture are multi-fold: It guides the network where to focus in the image and effectively encodes part constraints and context. More importantly, it can effectively cope with occlusions because part detection heatmaps for occluded parts provide low confidence scores which subsequently guide the regression part of our network to rely on contextual information in order to predict the location of these parts. Additionally, we show that the proposed cascade is flexible enough to readily allow the integration of various CNN architectures for both detection and regression, including recent ones based on residual learning. Finally, we illustrate that our cascade achieves top performance on the MPII and LSP data sets. Code can be downloaded from http://www.cs.nott.ac.uk/~psxab5/

accepted to ECCV 2016

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Fully Convolutional Networks for Semantic Segmentation
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Stacked Hourglass Networks for Human Pose Estimation
• Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations
• Convolutional Pose Machines
• Identity Mappings in Deep Residual Networks
• DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model
• Combining Local Appearance and Holistic View: Dual-Source Deep Neural Networks for Human Pose Estimation
• High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks
• Fine-grained pose prediction, normalization, and recognition
• Human Pose Estimation using Deep Consensus Voting
• Chained Predictions Using Convolutional Neural Networks

Cited by in corpus (102)

• A Survey of Deep Learning-based Object Detection
• OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• High-Resolution Representations for Labeling Pixels and Regions
• Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods
• Deep High-Resolution Representation Learning for Visual Recognition
• LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images
• Human Pose Regression by Combining Indirect Part Detection and Contextual Information
• Numerical Coordinate Regression with Convolutional Neural Networks
• Houdini: Fooling Deep Structured Prediction Models
• Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources
• Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition
• AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
• Cascaded Pyramid Network for Multi-Person Pose Estimation
• Rethinking on Multi-Stage Networks for Human Pose Estimation
• Region Ensemble Network: Improving Convolutional Network for Hand Pose Estimation
• Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• Deep Learning-Based Human Pose Estimation: A Survey
• Machine Learning: Algorithms, Models, and Applications
• Towards Accurate Multi-person Pose Estimation in the Wild
• Preterm infants' pose estimation with spatio-temporal features
• Robust Face Alignment by Multi-order High-precision Hourglass Network
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• T-LEAP: Occlusion-robust pose estimation of walking cows using temporal information
• Human Pose Estimation with Spatial Contextual Information
• Learning Feature Pyramids for Human Pose Estimation
• Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation
• Human Body Pose Estimation for Gait Identification: A Comprehensive Survey of Datasets and Models
• Deep High-Resolution Representation Learning for Human Pose Estimation
• Towards Good Practices for Deep 3D Hand Pose Estimation
• Two-stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge
• Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
• Improved training of binary networks for human pose estimation and image recognition
• Hierarchical binary CNNs for landmark localization with limited resources
• Self Adversarial Training for Human Pose Estimation
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression
• Real-time Convolutional Networks for Depth-based Human Pose Estimation
• Cascade Feature Aggregation for Human Pose Estimation
• Pixel-wise Regression: 3D Hand Pose Estimation via Spatial-form Representation and Differentiable Decoder
• Monocular Total Capture: Posing Face, Body, and Hands in the Wild
• Multi-Scale Structure-Aware Network for Human Pose Estimation
• Multi-Person Pose Estimation with Local Joint-to-Person Associations
• A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image
• Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views
• Deep Cuboid Detection: Beyond 2D Bounding Boxes
• ArtTrack: Articulated Multi-person Tracking in the Wild
• Integral Human Pose Regression
• Point detection through multi-instance deep heatmap regression for sutures in endoscopy
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• Holistic Planimetric prediction to Local Volumetric prediction for 3D Human Pose Estimation
• Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation
• Structure-Aware 3D Hourglass Network for Hand Pose Estimation from Single Depth Image
• Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization
• Pose Neural Fabrics Search
• Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement
• Quantized Densely Connected U-Nets for Efficient Landmark Localization
• In Situ Cane Toad Recognition
• SPCNet:Spatial Preserve and Content-aware Network for Human Pose Estimation
• A Detailed Look At CNN-based Approaches In Facial Landmark Detection
• DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image
• From Kinematics To Dynamics: Estimating Center of Pressure and Base of Support from Video Frames of Human Motion
• PoseTrack: Joint Multi-Person Pose Estimation and Tracking
• Lightweight Real-time Makeup Try-on in Mobile Browsers with Tiny CNN Models for Facial Tracking
• CoKe: Localized Contrastive Learning for Robust Keypoint Detection
• DeepSkeleton: Skeleton Map for 3D Human Pose Regression
• A Dual-Source Approach for 3D Human Pose Estimation from a Single Image
• Learning Dynamics from Kinematics: Estimating 2D Foot Pressure Maps from Video Frames
• AMIL: Adversarial Multi Instance Learning for Human Pose Estimation
• Multi-scale Aggregation R-CNN for 2D Multi-person Pose Estimation
• Matrix and tensor decompositions for training binary neural networks
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
• Anti-Confusing: Region-Aware Network for Human Pose Estimation
• Can Super Resolution be used to improve Human Pose Estimation in Low Resolution Scenarios?
• Fast Human Pose Estimation
• Single Person Pose Estimation: A Survey
• Patch-based 3D Human Pose Refinement
• Adversarial Semantic Data Augmentation for Human Pose Estimation
• Multi-Scale Networks for 3D Human Pose Estimation with Inference Stage Optimization
• Bi-directional Graph Structure Information Model for Multi-Person Pose Estimation
• 3D Human Pose Estimation Based on 2D-3D Consistency with Synchronized Adversarial Training
• Lightweight 3D Human Pose Estimation Network Training Using Teacher-Student Learning
• Mass Displacement Networks
• Multi-Person Pose Estimation with Enhanced Feature Aggregation and Selection
• Contact Area Detector using Cross View Projection Consistency for COVID-19 Projects
• CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis
• Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
• Pose estimator and tracker using temporal flow maps for limbs
• Measurement of Medial Elbow Joint Space using Landmark Detection
• Human Pose Estimation using Global and Local Normalization
• Feature Map Pooling for Cross-View Gait Recognition Based on Silhouette Sequence Images
• 3D Human Body Reconstruction from a Single Image via Volumetric Regression
• Semi- and Weakly-supervised Human Pose Estimation
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose
• MirrorNet: A Deep Bayesian Approach to Reflective 2D Pose Estimation from Human Images
• ORCEA: Object Recognition by Continuous Evidence Assimilation
• Combining Image- and Geometric-based Deep Learning for Shape Regression: A Comparison to Pixel-level Methods for Segmentation in Chest X-Ray
• Self-Supervision and Spatial-Sequential Attention Based Loss for Multi-Person Pose Estimation
• Boosting CNN-based primary quantization matrix estimation of double JPEG images via a classification-like architecture
• MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network
• Bottom-up Pose Estimation of Multiple Person with Bounding Box Constraint
• Joint Voxel and Coordinate Regression for Accurate 3D Facial Landmark Localization
• Hand-tremor frequency estimation in videos