ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters

Abstract

The incredible feats of athleticism demonstrated by humans are made possible in part by a vast repertoire of general-purpose motor skills, acquired through years of practice and experience. These skills not only enable humans to perform complex tasks, but also provide powerful priors for guiding their behaviors when learning new tasks. This is in stark contrast to what is common practice in physics-based character animation, where control policies are most typically trained from scratch for each task. In this work, we present a large-scale data-driven framework for learning versatile and reusable skill embeddings for physically simulated characters. Our approach combines techniques from adversarial imitation learning and unsupervised reinforcement learning to develop skill embeddings that produce life-like behaviors, while also providing an easy to control representation for use on new downstream tasks. Our models can be trained using large datasets of unstructured motion clips, without requiring any task-specific annotation or segmentation of the motion data. By leveraging a massively parallel GPU-based simulator, we are able to train skill embeddings using over a decade of simulated experiences, enabling our model to learn a rich and versatile repertoire of skills. We show that a single pre-trained model can be effectively applied to perform a diverse set of new tasks. Our system also allows users to specify tasks through simple reward functions, and the skill embedding then enables the character to automatically synthesize complex and naturalistic strategies in order to achieve the task objectives.

References in corpus (10)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Animating Human Athletics
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
• Character Controllers Using Motion VAEs
• Learning Symmetric and Low-energy Locomotion
• Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation
• CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion
• A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Control

Cited by in corpus (29)

• QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
• ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters
• CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
• C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
• HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation
• PADL: Language-Directed Physics-Based Character Control
• Contact-Implicit Model Predictive Control: Controlling Diverse Quadruped Motions Without Pre-Planned Contact Modes or Trajectories
• Composite Motion Learning with Task Control
• AdaptNet: Policy Adaptation for Physics-Based Character Control
• PDP: Physics-Based Character Animation via Diffusion Policy
• Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Control
• Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing
• Adaptive Tracking of a Single-Rigid-Body Character in Various Environments
• SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation
• PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers
• Physics-based Scene Layout Generation from Human Motion
• Learning to Walk and Fly with Adversarial Motion Priors
• PhysicsFC: Learning User-Controlled Skills for a Physics-Based Football Player Controller
• SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
• DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization
• CBIL: Collective Behavior Imitation Learning for Fish from Real Videos
• Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs
• Discovering Fatigued Movements for Virtual Character Animation
• Environment-aware Motion Matching
• ForceGrip: Reference-Free Curriculum Learning for Realistic Grip Force Control in VR Hand Manipulation
• CHOICE: Coordinated Human-Object Interaction in Cluttered Environments for Pick-and-Place Actions
• Learning to Ball: Composing Policies for Long-Horizon Basketball Moves
• ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild