TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors

Abstract

Motion capture is facing some new possibilities brought by the inertial sensing technologies which do not suffer from occlusion or wide-range recordings as vision-based solutions do. However, as the recorded signals are sparse and quite noisy, online performance and global translation estimation turn out to be two key difficulties. In this paper, we present TransPose, a DNN-based approach to perform full motion capture (with both global translations and body poses) from only 6 Inertial Measurement Units (IMUs) at over 90 fps. For body pose estimation, we propose a multi-stage network that estimates leaf-to-full joint positions as intermediate results. This design makes the pose estimation much easier, and thus achieves both better accuracy and lower computation cost. For global translation estimation, we propose a supporting-foot-based method and an RNN-based method to robustly solve for the global translations with a confidence-based fusion technique. Quantitative and qualitative comparisons show that our method outperforms the state-of-the-art learning- and optimization-based methods with a large margin in both accuracy and efficiency. As a purely inertial sensor-based approach, our method is not limited by environmental settings (e.g., fixed cameras), making the capture free from common difficulties such as wide-range motion space and strong occlusion.

arXiv commentAccepted by SIGGRAPH 2021. Project page: https://xinyu-yi.github.io/TransPose/

References in corpus (5)

• Sensor-based Gait Parameter Extraction with Deep Convolutional Neural Networks
• On the Continuity of Rotation Representations in Neural Networks
• Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS
• Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approach
• Monocular Real-time Full Body Capture with Inter-part Correlations

Cited by in corpus (6)

• QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
• IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds
• HOOV: Hand Out-Of-View Tracking for Proprioceptive Interaction using Inertial Sensing
• MobilePoser: Real-Time Full-Body Pose Estimation and 3D Human Translation from IMUs in Mobile Consumer Devices
• WheelPoser: Sparse-IMU Based Body Pose Estimation for Wheelchair Users
• Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective