A Survey on Deep Learning for Skeleton-Based Human Animation
arXiv:2110.06901 · doi:10.1111/cgf.14426
Abstract
Human character animation is often critical in entertainment content production, including video games, virtual reality or fiction films. To this end, deep neural networks drive most recent advances through deep learning and deep reinforcement learning. In this article, we propose a comprehensive survey on the state-of-the-art approaches based on either deep learning or deep reinforcement learning in skeleton-based human character animation. First, we introduce motion data representations, most common human motion datasets and how basic deep models can be enhanced to foster learning of spatial and temporal patterns in motion data. Second, we cover state-of-the-art approaches divided into three large families of applications in human animation pipelines: motion synthesis, character control and motion editing. Finally, we discuss the limitations of the current state-of-the-art methods based on deep learning and/or deep reinforcement learning in skeletal human character animation and possible directions of future research to alleviate current limitations and meet animators' needs.
arXiv comment32 pages, 10 figures, 4 tables, published in Computer Graphics Forum (CGF)
References in corpus (26)
- Auto-Encoding Variational Bayes
- Semi-Supervised Classification with Graph Convolutional Networks
- The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
- Geometric deep learning: going beyond Euclidean data
- NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
- NICE: Non-linear Independent Components Estimation
- DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
- Character Controllers Using Motion VAEs
- Robust Motion In-betweening
- Skeleton-Aware Networks for Deep Motion Retargeting
- QuaterNet: A Quaternion-based Recurrent Model for Human Motion
- MoGlow: Probabilistic and controllable motion synthesis using normalising flows
- Unpaired Motion Style Transfer from Video to Animation
- Learning Symmetric and Low-energy Locomotion
- Modeling Human Motion with Quaternion-based Neural Networks
- Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?
- MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
- Convolutional Autoencoders for Human Motion Infilling
- Neural probabilistic motor primitives for humanoid control
- Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation
- Hierarchical visuomotor control of humanoids
- LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signals
- Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic
- Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model
- DeepCap: Monocular Human Performance Capture Using Weak Supervision
- Machine Learning for Data-Driven Movement Generation: a Review of the State of the Art