DirectPose: Direct End-to-End Multi-Person Pose Estimation

Abstract

We propose the first direct end-to-end multi-person pose estimation framework, termed DirectPose. Inspired by recent anchor-free object detectors, which directly regress the two corners of target bounding-boxes, the proposed framework directly predicts instance-aware keypoints for all the instances from a raw input image, eliminating the need for heuristic grouping in bottom-up methods or bounding-box detection and RoI operations in top-down ones. We also propose a novel Keypoint Alignment (KPAlign) mechanism, which overcomes the main difficulty: lack of the alignment between the convolutional features and predictions in this end-to-end framework. KPAlign improves the framework's performance by a large margin while still keeping the framework end-to-end trainable. With the only postprocessing non-maximum suppression (NMS), our proposed framework can detect multi-person keypoints with or without bounding-boxes in a single shot. Experiments demonstrate that the end-to-end paradigm can achieve competitive or better performance than previous strong baselines, in both bottom-up and top-down methods. We hope that our end-to-end approach can provide a new perspective for the human pose estimation task.

12 pages

References in corpus (5)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• FCOS: Fully Convolutional One-Stage Object Detection
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• DenseBox: Unifying Landmark Localization with End to End Object Detection

Cited by in corpus (11)

• Deep Learning-Based Human Pose Estimation: A Survey
• TFPose: Direct Human Pose Estimation with Transformers
• FCOS: A simple and strong anchor-free object detector
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression
• Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation
• Pose Recognition with Cascade Transformers
• SiamMOT: Siamese Multi-Object Tracking
• FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions
• InsPose: Instance-Aware Networks for Single-Stage Multi-Person Pose Estimation
• Efficient Human Pose Estimation with Depthwise Separable Convolution and Person Centroid Guided Joint Grouping
• A Coarse-to-Fine Instance Segmentation Network with Learning Boundary Representation