C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters

Abstract

We present C$\cdot$ASE, an efficient and effective framework that learns conditional Adversarial Skill Embeddings for physics-based characters. Our physically simulated character can learn a diverse repertoire of skills while providing controllability in the form of direct manipulation of the skills to be performed. C$\cdot$ASE divides the heterogeneous skill motions into distinct subsets containing homogeneous samples for training a low-level conditional model to learn conditional behavior distribution. The skill-conditioned imitation learning naturally offers explicit control over the character's skills after training. The training course incorporates the focal skill sampling, skeletal residual forces, and element-wise feature masking to balance diverse skills of varying complexities, mitigate dynamics mismatch to master agile motions and capture more general behavior characteristics, respectively. Once trained, the conditional model can produce highly diverse and realistic skills, outperforming state-of-the-art models, and can be repurposed in various downstream tasks. In particular, the explicit skill control handle allows a high-level policy or user to direct the character with desired skill specifications, which we demonstrate is advantageous for interactive character animation.

arXiv commentSIGGRAPH Asia 2023

References in corpus (18)

• Learning Transferable Visual Models From Natural Language Supervision
• Categorical Reparameterization with Gumbel-Softmax
• NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
• Robust Motion In-betweening
• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters
• Human Motion Diffusion Model
• Neural probabilistic motor primitives for humanoid control
• ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters
• CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
• MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies
• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
• PADL: Language-Directed Physics-Based Character Control
• UniCon: Universal Neural Controller For Physics-based Character Motion
• TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
• Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction
• Interactive Character Control with Auto-Regressive Motion Diffusion Models

Cited by in corpus (6)

• PhysicsFC: Learning User-Controlled Skills for a Physics-Based Football Player Controller
• Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs
• AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning
• CBIL: Collective Behavior Imitation Learning for Fish from Real Videos
• Environment-aware Motion Matching
• ELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling