MANGA: Method Agnostic Neural-policy Generalization and Adaptation

Abstract

In this paper we target the problem of transferring policies across multiple environments with different dynamics parameters and motor noise variations, by introducing a framework that decouples the processes of policy learning and system identification. Efficiently transferring learned policies to an unknown environment with changes in dynamics configurations in the presence of motor noise is very important for operating robots in the real world, and our work is a novel attempt in that direction. We introduce MANGA: Method Agnostic Neural-policy Generalization and Adaptation, that trains dynamics conditioned policies and efficiently learns to estimate the dynamics parameters of the environment given off-policy state-transition rollouts in the environment. Our scheme is agnostic to the type of training method used - both reinforcement learning (RL) and imitation learning (IL) strategies can be used. We demonstrate the effectiveness of our approach by experimenting with four different MuJoCo agents and comparing against previously proposed transfer baselines.

arXiv commentUnder Review. Video available at https://drive.google.com/file/d/12GsDq3iQDXEutE-xpzXxqrEfD6dYhKjs/view?usp=sharing Other details will be made available in the author's webpage www.homangabharadhwaj.com

References in corpus (8)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Learning Dexterous In-Hand Manipulation
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
• The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems
• Preparing for the Unknown: Learning a Universal Policy with Online System Identification
• Decoupling Dynamics and Reward for Transfer Learning
• Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies