Modeling Human Motion with Quaternion-based Neural Networks

Abstract

Previous work on predicting or generating 3D human pose sequences regresses either joint rotations or joint positions. The former strategy is prone to error accumulation along the kinematic chain, as well as discontinuities when using Euler angles or exponential maps as parameterizations. The latter requires re-projection onto skeleton constraints to avoid bone stretching and invalid configurations. This work addresses both limitations. QuaterNet represents rotations with quaternions and our loss function performs forward kinematics on a skeleton to penalize absolute position errors instead of angle errors. We investigate both recurrent and convolutional architectures and evaluate on short-term prediction and long-term generation. For the latter, our approach is qualitatively judged as realistic as recent neural strategies from the graphics literature. Our experiments compare quaternions to Euler angles as well as exponential maps and show that only a very short context is required to make reliable future predictions. Finally, we show that the standard evaluation protocol for Human3.6M produces high variance results and we propose a simple solution.

arXiv commentFollow-up work of arXiv:1805.06485. This is a pre-print of an article published in IJCV. The final authenticated version is available online at https://doi.org/10.1007/s11263-019-01245-6

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Machine Translation by Jointly Learning to Align and Translate
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• On the difficulty of training Recurrent Neural Networks
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Language Modeling with Gated Convolutional Networks
• Sequence Level Training with Recurrent Neural Networks
• Deep multi-scale video prediction beyond mean square error
• Hands Deep in Deep Learning for Hand Pose Estimation
• QuaterNet: A Quaternion-based Recurrent Model for Human Motion
• Learning to Generate Long-term Future via Hierarchical Prediction
• Audio to Body Dynamics

Cited by in corpus (19)

• Skeleton-Aware Networks for Deep Motion Retargeting
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction
• Consistent 3D Hand Reconstruction in Video via self-supervised Learning
• A Survey on Deep Learning for Skeleton-Based Human Animation
• LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signals
• Real-time Controllable Motion Transition for Characters
• SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data
• Multitask Non-Autoregressive Model for Human Motion Prediction
• Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction
• Towards more realistic human motion prediction with attention to motion coordination
• UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons
• GroundLink: A Dataset Unifying Human Body Movement and Ground Reaction Dynamics
• Quaternion-Valued Convolutional Neural Network Applied for Acute Lymphoblastic Leukemia Diagnosis
• A causal convolutional neural network for multi-subject motion modeling and generation
• An Effective, Robust and Fairness-aware Hate Speech Detection Framework
• DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization
• Calculation of a force effect from muscle action to a quaternion-based musculoskeletal model
• Physics-constrained Attack against Convolution-based Human Motion Prediction
• Neural Networks at a Fraction with Pruned Quaternions