Cross-Domain Imitation Learning via Optimal Transport
arXiv:2110.03684
Abstract
Cross-domain imitation learning studies how to leverage expert demonstrations of one agent to train an imitation agent with a different embodiment or morphology. Comparing trajectories and stationary distributions between the expert and imitation agents is challenging because they live on different systems that may not even have the same dimensionality. We propose Gromov-Wasserstein Imitation Learning (GWIL), a method for cross-domain imitation that uses the Gromov-Wasserstein distance to align and compare states between the different spaces of the agents. Our theory formally characterizes the scenarios where GWIL preserves optimality, revealing its possibilities and limitations. We demonstrate the effectiveness of GWIL in non-trivial continuous control domains ranging from simple rigid transformation of the expert domain to arbitrary transformation of the state-action space.
ICLR 2022
References in corpus (15)
- Soft Actor-Critic Algorithms and Applications
- Dota 2 with Large Scale Deep Reinforcement Learning
- Emergence of Locomotion Behaviours in Rich Environments
- DeepMind Control Suite
- Third-Person Imitation Learning
- Reinforcement and Imitation Learning for Diverse Visuomotor Skills
- Towards Optimal Transport with Global Invariances
- Primal Wasserstein Imitation Learning
- State Alignment-based Imitation Learning
- Domain Adaptive Imitation Learning
- XIRL: Cross-embodiment Inverse Reinforcement Learning
- A contribution to Optimal Transport on incomparable spaces
- Hierarchically Decoupled Imitation for Morphological Transfer
- Imitation Learning with Sinkhorn Distances
- Cross-domain Imitation from Observations