Convolutional Sequence to Sequence Model for Human Dynamics

Abstract

Human motion modeling is a classic problem in computer vision and graphics. Challenges in modeling human motion include high dimensional prediction as well as extremely complicated dynamics.We present a novel approach to human motion modeling based on convolutional neural networks (CNN). The hierarchical structure of CNN makes it capable of capturing both spatial and temporal correlations effectively. In our proposed approach,a convolutional long-term encoder is used to encode the whole given motion sequence into a long-term hidden variable, which is used with a decoder to predict the remainder of the sequence. The decoder itself also has an encoder-decoder structure, in which the short-term encoder encodes a shorter sequence to a short-term hidden variable, and the spatial decoder maps the long and short-term hidden variable to motion predictions. By using such a model, we are able to capture both invariant and dynamic information of human motion, which results in more accurate predictions. Experiments show that our algorithm outperforms the state-of-the-art methods on the Human3.6M and CMU Motion Capture datasets. Our code is available at the project website.

CVPR2018

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Convolutional Sequence to Sequence Learning
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Neural Machine Translation in Linear Time
• Recurrent Network Models for Human Dynamics
• Learning Human Motion Models for Long-term Predictions

Cited by in corpus (4)

• Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies
• Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network
• BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
• Learning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks