AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control

Abstract

Synthesizing graceful and life-like behaviors for physically simulated characters has been a fundamental challenge in computer animation. Data-driven methods that leverage motion tracking are a prominent class of techniques for producing high fidelity motions for a wide range of behaviors. However, the effectiveness of these tracking-based methods often hinges on carefully designed objective functions, and when applied to large and diverse motion datasets, these methods require significant additional machinery to select the appropriate motion for the character to track in a given scenario. In this work, we propose to obviate the need to manually design imitation objectives and mechanisms for motion selection by utilizing a fully automated approach based on adversarial imitation learning. High-level task objectives that the character should perform can be specified by relatively simple reward functions, while the low-level style of the character's behaviors can be specified by a dataset of unstructured motion clips, without any explicit clip selection or sequencing. These motion clips are used to train an adversarial motion prior, which specifies style-rewards for training the character through reinforcement learning (RL). The adversarial RL procedure automatically selects which motion to perform, dynamically interpolating and generalizing from the dataset. Our system produces high-quality motions that are comparable to those achieved by state-of-the-art tracking-based techniques, while also being able to easily accommodate large datasets of unstructured motion clips. Composition of disparate skills emerges automatically from the motion prior, without requiring a high-level motion planner or other task-specific annotations of the motion clips. We demonstrate the effectiveness of our framework on a diverse cast of complex simulated characters and a challenging suite of motor control tasks.

References in corpus (19)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• End to End Learning for Self-Driving Cars
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Which Training Methods for GANs do actually Converge?
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• DeepMind Control Suite
• Character Controllers Using Motion VAEs
• Learning Symmetric and Low-energy Locomotion
• Learning human behaviors from motion capture by adversarial imitation
• Learning and Transfer of Modulated Locomotor Controllers
• Generative Adversarial Imitation from Observation
• Neural probabilistic motor primitives for humanoid control
• Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation
• CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion
• Imitation Learning as $f$-Divergence Minimization

Cited by in corpus (47)

• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters
• QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
• DTC: Deep Tracking Control
• InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions
• A Comprehensive Review of Data-Driven Co-Speech Gesture Generation
• Towards Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control
• ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters
• Transflower: probabilistic autoregressive dance generation with multimodal attention
• CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
• Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
• C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
• HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation
• From Screens to Scenes: A Survey of Embodied AI in Healthcare
• PADL: Language-Directed Physics-Based Character Control
• Contact-Implicit Model Predictive Control: Controlling Diverse Quadruped Motions Without Pre-Planned Contact Modes or Trajectories
• A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Control
• Composite Motion Learning with Task Control
• AdaptNet: Policy Adaptation for Physics-Based Character Control
• On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning
• Bidirectional GaitNet: A Bidirectional Prediction Model of Human Gait and Anatomical Conditions
• Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Control
• MAAIP: Multi-Agent Adversarial Interaction Priors for imitation from fighting demonstrations for physics-based characters
• Adaptive Tracking of a Single-Rigid-Body Character in Various Environments
• SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation
• PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers
• Physics-based Scene Layout Generation from Human Motion
• Learning to Walk and Fly with Adversarial Motion Priors
• Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization
• PhysicsFC: Learning User-Controlled Skills for a Physics-Based Football Player Controller
• Signs of Language: Embodied Sign Language Fingerspelling Acquisition from Demonstrations for Human-Robot Interaction
• AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning
• CBIL: Collective Behavior Imitation Learning for Fish from Real Videos
• SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
• Modeling human intention inference in continuous 3D domains by inverse planning and body kinematics
• Environment-aware Motion Matching
• Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs
• Implicit Behavioral Cloning
• Learning Human Search Behavior from Egocentric Visual Inputs
• Learning to Ball: Composing Policies for Long-Horizon Basketball Moves
• ForceGrip: Reference-Free Curriculum Learning for Realistic Grip Force Control in VR Hand Manipulation
• CHOICE: Coordinated Human-Object Interaction in Cluttered Environments for Pick-and-Place Actions
• Manipulate as Human: Learning Task-oriented Manipulation Skills by Adversarial Motion Priors
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild
• HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario
• Advances, challenges, and opportunities for legged robots
• Perception-and-action system for humanoid robot task execution in construction
• Learning High-Risk High-Precision Motion Control