Towards Accurate Multi-person Pose Estimation in the Wild

Abstract

We propose a method for multi-person detection and 2-D pose estimation that achieves state-of-art results on the challenging COCO keypoints task. It is a simple, yet powerful, top-down approach consisting of two stages. In the first stage, we predict the location and scale of boxes which are likely to contain people; for this we use the Faster RCNN detector. In the second stage, we estimate the keypoints of the person potentially contained in each proposed bounding box. For each keypoint type we predict dense heatmaps and offsets using a fully convolutional ResNet. To combine these outputs we introduce a novel aggregation procedure to obtain highly localized keypoint predictions. We also use a novel form of keypoint-based Non-Maximum-Suppression (NMS), instead of the cruder box-level NMS, and a novel form of keypoint-based confidence score estimation, instead of box-level scoring. Trained on COCO data alone, our final system achieves average precision of 0.649 on the COCO test-dev set and the 0.643 test-standard sets, outperforming the winner of the 2016 COCO keypoints challenge and other recent state-of-art. Further, by using additional in-house labeled data we obtain an even higher average precision of 0.685 on the test-dev set and 0.673 on the test-standard set, more than 5% absolute improvement compared to the previous best performing method on the same dataset.

arXiv commentPaper describing an improved version of the G-RMI entry to the 2016 COCO keypoints challenge (http://image-net.org/challenges/ilsvrc+coco2016). Camera ready version to appear in the Proceedings of CVPR 2017

References in corpus (11)

• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• DeepPose: Human Pose Estimation via Deep Neural Networks
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Human pose estimation via Convolutional Part Heatmap Regression
• Stacked Hourglass Networks for Human Pose Estimation
• Convolutional Pose Machines
• Learning Human Pose Estimation Features with Convolutional Networks
• Speed/accuracy trade-offs for modern convolutional object detectors
• Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• Multi-Person Pose Estimation with Local Joint-to-Person Associations
• ArtTrack: Articulated Multi-person Tracking in the Wild

Cited by in corpus (32)

• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Simple Baselines for Human Pose Estimation and Tracking
• TFPose: Direct Human Pose Estimation with Transformers
• Pose Trainer: Correcting Exercise Posture using Pose Estimation
• Decoupled Classification Refinement: Hard False Positive Suppression for Object Detection
• Heatmap Regression via Randomized Rounding
• Learning Human-Object Interaction Detection using Interaction Points
• Joint Flow: Temporal Flow Fields for Multi Person Tracking
• Single-Stage Multi-Person Pose Machines
• Integral Human Pose Regression
• Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation
• Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation
• The Intelligent ICU Pilot Study: Using Artificial Intelligence Technology for Autonomous Patient Monitoring
• Generative Partition Networks for Multi-Person Pose Estimation
• An Integral Pose Regression System for the ECCV2018 PoseTrack Challenge
• Dual Path Networks for Multi-Person Human Pose Estimation
• Learning to Refine Human Pose Estimation
• PERF-Net: Pose Empowered RGB-Flow Net
• Multi-person Articulated Tracking with Spatial and Temporal Embeddings
• Pose-Based Two-Stream Relational Networks for Action Recognition in Videos
• Combining detection and tracking for human pose estimation in videos
• Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing
• Whole-Body Human Pose Estimation in the Wild
• Multi-scale Aggregation R-CNN for 2D Multi-person Pose Estimation
• When Vehicles See Pedestrians with Phones:A Multi-Cue Framework for Recognizing Phone-based Activities of Pedestrians
• Bi-directional Graph Structure Information Model for Multi-Person Pose Estimation
• Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
• Deep Reinforcement Learning for Active Human Pose Estimation
• Simple Multi-Resolution Representation Learning for Human Pose Estimation
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose
• Bottom-up Pose Estimation of Multiple Person with Bounding Box Constraint
• Detector-in-Detector: Multi-Level Analysis for Human-Parts