Human Pose Estimation from Ambiguous Pressure Recordings with Spatio-temporal Masked Transformers

Abstract

Despite the impressive performance of vision-based pose estimators, they generally fail to perform well under adverse vision conditions and often don't satisfy the privacy demands of customers. As a result, researchers have begun to study tactile sensing systems as an alternative. However, these systems suffer from noisy and ambiguous recordings. To tackle this problem, we propose a novel solution for pose estimation from ambiguous pressure data. Our method comprises a spatio-temporal vision transformer with an encoder-decoder architecture. Detailed experiments on two popular public datasets reveal that our model outperforms existing solutions in the area. Moreover, we observe that increasing the number of temporal crops in the early stages of the network positively impacts the performance while pre-training the network in a self-supervised setting using a masked auto-encoder approach also further improves the results.

References in corpus (6)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• On the Convergence of Adam and Beyond
• ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation
• Identity and Posture Recognition in Smart Beds with Deep Multitask Learning
• Estimating Pose from Pressure Data for Smart Beds with Deep Image-based Pose Estimators
• Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditions