Character Controllers Using Motion VAEs

Abstract

A fundamental problem in computer animation is that of realizing purposeful and realistic human movement given a sufficiently-rich set of motion capture clips. We learn data-driven generative models of human movement using autoregressive conditional variational autoencoders, or Motion VAEs. The latent variables of the learned autoencoder define the action space for the movement and thereby govern its evolution over time. Planning or control algorithms can then use this action space to generate desired motions. In particular, we use deep reinforcement learning to learn controllers that achieve goal-directed movements. We demonstrate the effectiveness of the approach on multiple tasks. We further evaluate system-design choices and describe the current limitations of Motion VAEs.

Project page: https://www.cs.ubc.ca/~hyuling/projects/mvae/ ; Code: https://github.com/electronicarts/character-motion-vaes

References in corpus (2)

• Emergence of Locomotion Behaviours in Rich Environments
• Meta Learning Shared Hierarchies

Cited by in corpus (14)

• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters
• QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
• ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters
• CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
• Real-time Controllable Motion Transition for Characters
• Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors
• PADL: Language-Directed Physics-Based Character Control
• Composite Motion Learning with Task Control
• AdaptNet: Policy Adaptation for Physics-Based Character Control
• We are More than Our Joints: Predicting how 3D Bodies Move
• Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders
• A Survey on Reinforcement Learning Methods in Character Animation
• Task-Generic Hierarchical Human Motion Prior using VAEs
• Deep Generative Modelling of Human Reach-and-Place Action