Cascaded Pyramid Network for Multi-Person Pose Estimation

Abstract

The topic of multi-person pose estimation has been largely improved recently, especially with the development of convolutional neural network. However, there still exist a lot of challenging cases, such as occluded keypoints, invisible keypoints and complex background, which cannot be well addressed. In this paper, we present a novel network structure called Cascaded Pyramid Network (CPN) which targets to relieve the problem from these "hard" keypoints. More specifically, our algorithm includes two stages: GlobalNet and RefineNet. GlobalNet is a feature pyramid network which can successfully localize the "simple" keypoints like eyes and hands but may fail to precisely recognize the occluded or invisible keypoints. Our RefineNet tries explicitly handling the "hard" keypoints by integrating all levels of feature representations from the GlobalNet together with an online hard keypoint mining loss. In general, to address the multi-person pose estimation problem, a top-down pipeline is adopted to first generate a set of human bounding boxes based on a detector, followed by our CPN for keypoint localization in each human bounding box. Based on the proposed algorithm, we achieve state-of-art results on the COCO keypoint benchmark, with average precision at 73.0 on the COCO test-dev dataset and 72.1 on the COCO test-challenge dataset, which is a 19% relative improvement compared with 60.5 from the COCO 2016 keypoint challenge.Code (https://github.com/chenyilun95/tf-cpn.git) and the detection results are publicly available for further research.

10 pages, accepted to CVPR 2018

Cited by in corpus (23)

• Deep High-Resolution Representation Learning for Visual Recognition
• Deep High-Resolution Representation Learning for Human Pose Estimation
• TFPose: Direct Human Pose Estimation with Transformers
• CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark
• Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views
• Action Machine: Rethinking Action Recognition in Trimmed Videos
• ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving
• Pose2Seg: Detection Free Human Instance Segmentation
• LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking
• Improving Multi-Person Pose Estimation using Label Correction
• Learning to Refine Human Pose Estimation
• A Context-and-Spatial Aware Network for Multi-Person Pose Estimation
• Deep Dose Plugin Towards Real-time Monte Carlo Dose Calculation Through a Deep Learning based Denoising Algorithm
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation
• Deep Feature Pyramid Reconfiguration for Object Detection
• Two-Stream Video Classification with Cross-Modality Attention
• The Influence of Faulty Labels in Data Sets on Human Pose Estimation
• Weakly Supervised Keypoint Discovery
• A Tracking System For Baseball Game Reconstruction
• Detector-in-Detector: Multi-Level Analysis for Human-Parts
• A Global to Local Double Embedding Method for Multi-person Pose Estimation
• HANDS18: Methods, Techniques and Applications for Hand Observation
• MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network