Pose Recognition with Cascade Transformers

Abstract

In this paper, we present a regression-based pose recognition method using cascade Transformers. One way to categorize the existing approaches in this domain is to separate them into 1). heatmap-based and 2). regression-based. In general, heatmap-based methods achieve higher accuracy but are subject to various heuristic designs (not end-to-end mostly), whereas regression-based approaches attain relatively lower accuracy but they have less intermediate non-differentiable steps. Here we utilize the encoder-decoder structure in Transformers to perform regression-based person and keypoint detection that is general-purpose and requires less heuristic design compared with the existing approaches. We demonstrate the keypoint hypothesis (query) refinement process across different self-attention layers to reveal the recursive self-attention mechanism in Transformers. In the experiments, we report competitive results for pose recognition when compared with the competing regression-based methods.

arXiv commentAccepted to CVPR 2021

References in corpus (7)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Decoupled Weight Decay Regularization
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• DirectPose: Direct End-to-End Multi-Person Pose Estimation
• The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation

Cited by in corpus (1)

• TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation