TokenPose: Learning Keypoint Tokens for Human Pose Estimation

Abstract

Human pose estimation deeply relies on visual clues and anatomical constraints between parts to locate keypoints. Most existing CNN-based methods do well in visual representation, however, lacking in the ability to explicitly learn the constraint relationships between keypoints. In this paper, we propose a novel approach based on Token representation for human Pose estimation~(TokenPose). In detail, each keypoint is explicitly embedded as a token to simultaneously learn constraint relationships and appearance cues from images. Extensive experiments show that the small and large TokenPose models are on par with state-of-the-art CNN-based counterparts while being more lightweight. Specifically, our TokenPose-S and TokenPose-L achieve $72.5$ AP and $75.8$ AP on COCO validation dataset respectively, with significant reduction in parameters ($\downarrow 80.6\%$; $\downarrow$ $56.8\%$) and GFLOPs ($\downarrow$ $75.3\%$; $\downarrow$ $24.7\%$). Code is publicly available.

arXiv commentAccepted by ICCV'21. Code is publicly available at https://github.com/leeyegy/TokenPose

References in corpus (11)

• Efficient Estimation of Word Representations in Vector Space
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Language Models are Few-Shot Learners
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Transformer in Transformer
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Pre-Trained Image Processing Transformer
• How to Start Training: The Effect of Initialization and Architecture
• End-to-End Human Pose and Mesh Reconstruction with Transformers
• End-to-End Trainable Multi-Instance Pose Estimation with Transformers

Cited by in corpus (1)

• SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation