MoVi: A Large Multipurpose Motion and Video Dataset

Abstract

Human movements are both an area of intense study and the basis of many applications such as character animation. For many applications, it is crucial to identify movements from videos or analyze datasets of movements. Here we introduce a new human Motion and Video dataset MoVi, which we make available publicly. It contains 60 female and 30 male actors performing a collection of 20 predefined everyday actions and sports movements, and one self-chosen movement. In five capture rounds, the same actors and movements were recorded using different hardware systems, including an optical motion capture system, video cameras, and inertial measurement units (IMU). For some of the capture rounds, the actors were recorded when wearing natural clothing, for the other rounds they wore minimal clothing. In total, our dataset contains 9 hours of motion capture data, 17 hours of video data from 4 different points of view (including one hand-held camera), and 6.6 hours of IMU data. In this paper, we describe how the dataset was collected and post-processed; We present state-of-the-art estimates of skeletal motions and full-body shape deformations associated with skeletal motion. We discuss examples for potential studies this dataset could enable.

References in corpus (6)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Spatiotemporal Residual Networks for Video Action Recognition
• Deep High-Resolution Representation Learning for Human Pose Estimation
• Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model
• RoboNet: Large-Scale Multi-Robot Learning

Cited by in corpus (7)

• Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model
• Towards Single Camera Human 3D-Kinematics
• Motion Capture Benchmark of Real Industrial Tasks and Traditional Crafts for Human Movement Analysis
• Berkeley Open Extended Reality Recordings 2023 (BOXRR-23): 4.7 Million Motion Capture Recordings from 105,852 Extended Reality Device Users
• Machine Learning in Biomechanics: Key Applications and Limitations in Walking, Running, and Sports Movements
• Fusing uncalibrated IMUs and handheld smartphone video to reconstruct knee kinematics
• SpATr: MoCap 3D Human Action Recognition based on Spiral Auto-encoder and Transformer Network