QuaterNet: A Quaternion-based Recurrent Model for Human Motion

Abstract

Deep learning for predicting or generating 3D human pose sequences is an active research area. Previous work regresses either joint rotations or joint positions. The former strategy is prone to error accumulation along the kinematic chain, as well as discontinuities when using Euler angle or exponential map parameterizations. The latter requires re-projection onto skeleton constraints to avoid bone stretching and invalid configurations. This work addresses both limitations. Our recurrent network, QuaterNet, represents rotations with quaternions and our loss function performs forward kinematics on a skeleton to penalize absolute position errors instead of angle errors. On short-term predictions, QuaterNet improves the state-of-the-art quantitatively. For long-term generation, our approach is qualitatively judged as realistic as recent neural strategies from the graphics literature.

arXiv commentBritish Machine Vision Conference (BMVC), 2018

References in corpus (4)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Pixel Recurrent Neural Networks

Cited by in corpus (33)

• Analyzing Input and Output Representations for Speech-Driven Gesture Generation
• Modeling Human Motion with Quaternion-based Neural Networks
• MoVi: A Large Multipurpose Motion and Video Dataset
• MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
• Convolutional Autoencoders for Human Motion Infilling
• Transflower: probabilistic autoregressive dance generation with multimodal attention
• A Survey on Deep Learning for Skeleton-Based Human Animation
• LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signals
• Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure
• SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data
• Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation
• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
• Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction
• HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation
• Generating Emotive Gaits for Virtual Agents Using Affect-Based Autoregression
• DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
• The Liar's Walk: Detecting Deception with Gait and Gesture
• Pose Representations for Deep Skeletal Animation
• PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction
• Learning Variations in Human Motion via Mix-and-Match Perturbation
• Motion Prediction Using Temporal Inception Module
• Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction
• KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints
• An Effective, Robust and Fairness-aware Hate Speech Detection Framework
• Deep learning-based estimation of whole-body kinematics from multi-view images
• Economical Quaternion Extraction from a Human Skeletal Pose Estimate using 2-D Cameras
• Quaternion Product Units for Deep Learning on 3D Rotation Groups
• DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization
• EWareNet: Emotion Aware Human Intent Prediction and Adaptive Spatial Profile Fusion for Social Robot Navigation
• Learning Multiscale Correlations for Human Motion Prediction
• Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic
• Hierarchical Graph-Convolutional Variational AutoEncoding for Generative Modelling of Human Motion