ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters

Abstract

In this paper, we introduce ControlVAE, a novel model-based framework for learning generative motion control policies based on variational autoencoders (VAE). Our framework can learn a rich and flexible latent representation of skills and a skill-conditioned generative control policy from a diverse set of unorganized motion sequences, which enables the generation of realistic human behaviors by sampling in the latent space and allows high-level control policies to reuse the learned skills to accomplish a variety of downstream tasks. In the training of ControlVAE, we employ a learnable world model to realize direct supervision of the latent space and the control policy. This world model effectively captures the unknown dynamics of the simulation system, enabling efficient model-based learning of high-level downstream tasks. We also learn a state-conditional prior distribution in the VAE-based generative control policy, which generates a skill embedding that outperforms the non-conditional priors in downstream tasks. We demonstrate the effectiveness of ControlVAE using a diverse set of tasks, which allows realistic and interactive control of the simulated characters.

arXiv commentSIGGRAPH Asia 2022 (Journal Track);

References in corpus (10)

• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
• Character Controllers Using Motion VAEs
• Robust Motion In-betweening
• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters
• Unpaired Motion Style Transfer from Video to Animation
• MoGlow: Probabilistic and controllable motion synthesis using normalising flows
• Learning Symmetric and Low-energy Locomotion
• GANimator: Neural Motion Synthesis from a Single Sequence
• CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion

Cited by in corpus (14)

• C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
• AdaptNet: Policy Adaptation for Physics-Based Character Control
• PDP: Physics-Based Character Animation via Diffusion Policy
• Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Control
• Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing
• Adaptive Tracking of a Single-Rigid-Body Character in Various Environments
• SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation
• PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers
• Physics-based Scene Layout Generation from Human Motion
• PhysicsFC: Learning User-Controlled Skills for a Physics-Based Football Player Controller
• CBIL: Collective Behavior Imitation Learning for Fish from Real Videos
• AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning
• Discovering Fatigued Movements for Virtual Character Animation
• Learning to Ball: Composing Policies for Long-Horizon Basketball Moves