Test-Time Personalization with a Transformer for Human Pose Estimation
arXiv:2107.02133
Abstract
We propose to personalize a human pose estimator given a set of test images of a person without using any manual annotations. While there is a significant advancement in human pose estimation, it is still very challenging for a model to generalize to different unknown environments and unseen persons. Instead of using a fixed model for every test case, we adapt our pose estimator during test time to exploit person-specific information. We first train our model on diverse data with both a supervised and a self-supervised pose estimation objectives jointly. We use a Transformer model to build a transformation between the self-supervised keypoints and the supervised keypoints. During test time, we personalize and adapt our model by fine-tuning with the self-supervised objective. The pose is then improved by transforming the updated self-supervised keypoints. We experiment with multiple datasets and show significant improvements on pose estimations with our self-supervised personalization.
arXiv commentProject page: http://liyz15.github.io/TTP/
References in corpus (8)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- DeepPose: Human Pose Estimation via Deep Neural Networks
- Tent: Fully Test-time Adaptation by Entropy Minimization
- Stand-Alone Self-Attention in Vision Models
- Unsupervised Learning of Object Landmarks through Conditional Image Generation
- TFPose: Direct Human Pose Estimation with Transformers
- Online Adaptation for Consistent Mesh Reconstruction in the Wild
- End-to-End Trainable Multi-Instance Pose Estimation with Transformers