Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

Abstract

This paper proposes a new hybrid architecture that consists of a deep Convolutional Network and a Markov Random Field. We show how this architecture is successfully applied to the challenging problem of articulated human pose estimation in monocular images. The architecture can exploit structural domain constraints such as geometric relationships between body joint locations. We show that joint training of these two model paradigms improves performance and allows us to significantly outperform existing state-of-the-art techniques.

References in corpus (4)

• DeepPose: Human Pose Estimation via Deep Neural Networks
• Fast Training of Convolutional Networks through FFTs
• Learning Human Pose Estimation Features with Convolutional Networks
• Fast Image Scanning with Deep Max-Pooling Convolutional Neural Networks

Cited by in corpus (137)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Conditional Random Fields as Recurrent Neural Networks
• Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• Human pose estimation via Convolutional Part Heatmap Regression
• Deep learning tools for the measurement of animal behavior in neuroscience
• Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods
• GLAD: Global-Local-Alignment Descriptor for Pedestrian Retrieval
• Integrating Spatial Configuration into Heatmap Regression Based CNNs for Landmark Localization
• A Comprehensive Analysis of Deep Regression
• Hands Deep in Deep Learning for Hand Pose Estimation
• LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• MIT Advanced Vehicle Technology Study: Large-Scale Naturalistic Driving Study of Driver Behavior and Interaction with Automation
• Deep Pictorial Gaze Estimation
• Synthetic Depth-of-Field with a Single-Camera Mobile Phone
• Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings
• PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
• AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
• Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
• MoVi: A Large Multipurpose Motion and Video Dataset
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Learning Deep Structured Models
• Vision-Based Human Pose Estimation via Deep Learning: A Survey
• Pedestrian Models for Autonomous Driving Part I: Low-Level Models, from Sensing to Tracking
• Deep Learning-Based Human Pose Estimation: A Survey
• Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection
• Improving Deep Learning for HAR with shallow LSTMs
• Synthesizing Training Images for Boosting Human 3D Pose Estimation
• Human Pose Estimation with Spatial Contextual Information
• Learning Transferable Parameters for Unsupervised Domain Adaptation
• Human Body Pose Estimation for Gait Identification: A Comprehensive Survey of Datasets and Models
• Deep High-Resolution Representation Learning for Human Pose Estimation
• Automatic Building Extraction in Aerial Scenes Using Convolutional Networks
• Heatmap Regression via Randomized Rounding
• Distribution-Aware Coordinate Representation for Human Pose Estimation
• Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
• Improved training of binary networks for human pose estimation and image recognition
• Hierarchical binary CNNs for landmark localization with limited resources
• Expanded Parts Model for Semantic Description of Humans in Still Images
• Simple and Lightweight Human Pose Estimation
• TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation
• Enhancing Energy Minimization Framework for Scene Text Recognition with Top-Down Cues
• The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation
• FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks
• SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation
• Real-time Convolutional Networks for Depth-based Human Pose Estimation
• CREIME: A Convolutional Recurrent model for Earthquake Identification and Magnitude Estimation
• Deeply Learning the Messages in Message Passing Inference
• EvoPose2D: Pushing the Boundaries of 2D Human Pose Estimation using Accelerated Neuroevolution with Weight Transfer
• Emergence of Cooperation in Two-agent Repeated Games with Reinforcement Learning
• Discriminative Training of Deep Fully-connected Continuous CRF with Task-specific Loss
• Multi-Scale Structure-Aware Network for Human Pose Estimation
• Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints
• Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation
• Deep Learning in Bioinformatics
• Enhancing Human Pose Estimation in Ancient Vase Paintings via Perceptually-grounded Style Transfer Learning
• Optimal Pose and Shape Estimation for Category-level 3D Object Perception
• DanceIt: Music-inspired Dancing Video Synthesis
• Adaptive Graphical Model Network for 2D Handpose Estimation
• 3D Interpreter Networks for Viewer-Centered Wireframe Modeling
• DeepMark: One-Shot Clothing Detection
• Person-in-WiFi: Fine-grained Person Perception using WiFi
• Learning Temporal Pose Estimation from Sparsely-Labeled Videos
• Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation
• Profile approach for recognition of three-dimensional magnetic structures
• Pose from Action: Unsupervised Learning of Pose Features based on Motion
• A Mathematical Analysis of Learning Loss for Active Learning in Regression
• Exploiting Offset-guided Network for Pose Estimation and Tracking
• 3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training
• Structure-Aware 3D Hourglass Network for Hand Pose Estimation from Single Depth Image
• Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization
• AID: Pushing the Performance Boundary of Human Pose Estimation with Information Dropping Augmentation
• Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement
• Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networks
• Pose Neural Fabrics Search
• Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models
• Attribute And-Or Grammar for Joint Parsing of Human Attributes, Part and Pose
• Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation
• In Situ Cane Toad Recognition
• DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image
• From Kinematics To Dynamics: Estimating Center of Pressure and Base of Support from Video Frames of Human Motion
• Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation
• Global Context for Convolutional Pose Machines
• On the Robustness of Human Pose Estimation
• AMIL: Adversarial Multi Instance Learning for Human Pose Estimation
• Learning Dynamics from Kinematics: Estimating 2D Foot Pressure Maps from Video Frames
• Multi-scale Aggregation R-CNN for 2D Multi-person Pose Estimation
• TRB: A Novel Triplet Representation for Understanding 2D Human Body
• Multi-person 3D pose estimation from unlabelled data
• Context Modeling in 3D Human Pose Estimation: A Unified Perspective
• Skeleton Based Action Recognition using a Stacked Denoising Autoencoder with Constraints of Privileged Information
• Fast Human Pose Estimation
• Can Super Resolution be used to improve Human Pose Estimation in Low Resolution Scenarios?
• Automatic Analysis System of Calcaneus Radiograph: Rotation-Invariant Landmark Detection for Calcaneal Angle Measurement, Fracture Identification and Fracture Region Segmentation
• Integrated Inference and Learning of Neural Factors in Structural Support Vector Machines
• Adversarial Semantic Data Augmentation for Human Pose Estimation
• Single Person Pose Estimation: A Survey
• Believe It or Not, We Know What You Are Looking at!
• Using deep neural networks to improve the precision of fast-sampled particle timing detectors
• Composite Localization for Human Pose Estimation
• Video Based Fall Detection Using Human Poses
• Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation
• Humanoid: A Deep Learning-based Approach to Automated Black-box Android App Testing
• Multi-Scale Networks for 3D Human Pose Estimation with Inference Stage Optimization
• Spine Landmark Localization with combining of Heatmap Regression and Direct Coordinate Regression
• Deep Global-Connected Net With The Generalized Multi-Piecewise ReLU Activation in Deep Learning
• Multi-Person Pose Estimation with Enhanced Feature Aggregation and Selection
• Joint COCO and Mapillary Workshop at ICCV 2019 Keypoint Detection Challenge Track Technical Report: Distribution-Aware Coordinate Representation for Human Pose Estimation
• In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representations
• Regressive Domain Adaptation for Unsupervised Keypoint Detection
• SpineOne: A One-Stage Detection Framework for Degenerative Discs and Vertebrae
• Pose estimator and tracker using temporal flow maps for limbs
• Single View Physical Distance Estimation using Human Pose
• PAC-GAN: An Effective Pose Augmentation Scheme for Unsupervised Cross-View Person Re-identification
• Differentiable Nonparametric Belief Propagation
• Scaling Matters in Deep Structured-Prediction Models
• Deep Markov Random Field for Image Modeling
• Oscillatory evolution of collective behavior in evolutionary games played with reinforcement learning
• Reciprocal Landmark Detection and Tracking with Extremely Few Annotations
• Semi- and Weakly-supervised Human Pose Estimation
• MirrorNet: A Deep Bayesian Approach to Reflective 2D Pose Estimation from Human Images
• Towards Viewpoint Invariant 3D Human Pose Estimation
• An Internal Covariate Shift Bounding Algorithm for Deep Neural Networks by Unitizing Layers' Outputs
• MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network
• A Unified Framework of Surrogate Loss by Refactoring and Interpolation
• Multi-Scale Supervised Network for Human Pose Estimation
• Object Parsing in Sequences Using CoordConv Gated Recurrent Networks
• Adversarial Synthesis of Human Pose from Text
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose
• Using deep neural networks to detect non-analytically defined expert event labels in canoe sprint force sensor signals
• Self-Supervision and Spatial-Sequential Attention Based Loss for Multi-Person Pose Estimation
• Semantic-based End-to-End Learning for Typhoon Intensity Prediction
• Improved Descriptors for Patch Matching and Reconstruction