MobilePoser: Real-Time Full-Body Pose Estimation and 3D Human Translation from IMUs in Mobile Consumer Devices

Abstract

There has been a continued trend towards minimizing instrumentation for full-body motion capture, going from specialized rooms and equipment, to arrays of worn sensors and recently sparse inertial pose capture methods. However, as these techniques migrate towards lower-fidelity IMUs on ubiquitous commodity devices, like phones, watches, and earbuds, challenges arise including compromised online performance, temporal consistency, and loss of global translation due to sensor noise and drift. Addressing these challenges, we introduce MobilePoser, a real-time system for full-body pose and global translation estimation using any available subset of IMUs already present in these consumer devices. MobilePoser employs a multi-stage deep neural network for kinematic pose estimation followed by a physics-based motion optimizer, achieving state-of-the-art accuracy while remaining lightweight. We conclude with a series of demonstrative applications to illustrate the unique potential of MobilePoser across a variety of fields, such as health and wellness, gaming, and indoor navigation to name a few.

References in corpus (5)

• IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds
• Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation
• TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors
• SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data
• SmartPoser: Arm Pose Estimation with a Smartphone and Smartwatch Using UWB and IMU Data

Cited by in corpus (3)

• WatchHAR: Real-time On-device Human Activity Recognition System for Smartwatches
• IMUCoCo: Enabling Flexible On-Body IMU Placement for Human Pose Estimation and Activity Recognition
• BaroPoser: Real-time Human Motion Tracking from IMUs and Barometers in Everyday Devices