Learning Human Motion Models for Long-term Predictions

Abstract

We propose a new architecture for the learning of predictive spatio-temporal motion models from data alone. Our approach, dubbed the Dropout Autoencoder LSTM, is capable of synthesizing natural looking motion sequences over long time horizons without catastrophic drift or motion degradation. The model consists of two components, a 3-layer recurrent neural network to model temporal aspects and a novel auto-encoder that is trained to implicitly recover the spatial structure of the human skeleton via randomly removing information about joints during training time. This Dropout Autoencoder (D-AE) is then used to filter each predicted pose of the LSTM, reducing accumulation of error and hence drift over time. Furthermore, we propose new evaluation protocols to assess the quality of synthetic motion sequences even for which no ground truth data exists. The proposed protocols can be used to assess generated sequences of arbitrary length. Finally, we evaluate our proposed method on two of the largest motion-capture datasets available to date and show that our model outperforms the state-of-the-art on a variety of actions, including cyclic and acyclic motion, and that it can produce natural looking sequences over longer time horizons than previous methods.

References in corpus (7)

• Conditional Image Synthesis With Auxiliary Classifier GANs
• NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
• Show and Tell: A Neural Image Caption Generator
• Differential Recurrent Neural Networks for Action Recognition
• Structured Prediction of 3D Human Pose with Deep Neural Networks
• Space-Time Representation of People Based on 3D Skeletal Data: A Review
• Anticipating many futures: Online human motion prediction and synthesis for human-robot collaboration

Cited by in corpus (6)

• Robust Motion In-betweening
• Convolutional Sequence to Sequence Model for Human Dynamics
• Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction
• Imitation Learning for Human Pose Prediction
• BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
• A neural network based post-filter for speech-driven head motion synthesis