SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning
arXiv:2101.06005
Abstract
As learning-based approaches progress towards automating robot controllers design, transferring learned policies to new domains with different dynamics (e.g. sim-to-real transfer) still demands manual effort. This paper introduces SimGAN, a framework to tackle domain adaptation by identifying a hybrid physics simulator to match the simulated trajectories to the ones from the target domain, using a learned discriminative loss to address the limitations associated with manual loss design. Our hybrid simulator combines neural networks and traditional physics simulation to balance expressiveness and generalizability, and alleviates the need for a carefully selected parameter set in System ID. Once the hybrid simulator is identified via adversarial reinforcement learning, it can be used to refine policies for the target domain, without the need to interleave data collection and policy refinement. We show that our approach outperforms multiple strong baselines on six robotic locomotion tasks for domain adaptation.
arXiv commentICRA 2021, Code Available at: https://github.com/jyf588/SimGAN ; Accompanying Video: https://youtu.be/McKOGllO7nc
References in corpus (11)
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Learning agile and dynamic motor skills for legged robots
- Solving Rubik's Cube with a Robot Hand
- Robust Adversarial Reinforcement Learning
- EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
- Sim-to-Real Robot Learning from Pixels with Progressive Nets
- Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
- Active Domain Randomization
- Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfer
- Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers
- Learning State-Dependent Losses for Inverse Dynamics Learning