Robust Reinforcement Learning via Adversarial training with Langevin Dynamics
arXiv:2002.06063
Abstract
We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents. Leveraging the powerful Stochastic Gradient Langevin Dynamics, we present a novel, scalable two-player RL algorithm, which is a sampling variant of the two-player policy gradient method. Our algorithm consistently outperforms existing baselines, in terms of generalization across different training and testing conditions, on several MuJoCo environments. Our experiments also show that, even for objective functions that entirely ignore potential environmental shifts, our sampling approach remains highly robust in comparison to standard RL algorithms.
References in corpus (12)
- Continuous control with deep reinforcement learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Trust Region Policy Optimization
- Addressing Function Approximation Error in Actor-Critic Methods
- Robust Adversarial Reinforcement Learning
- A Variational Inequality Perspective on Generative Adversarial Networks
- Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
- Stein Variational Policy Gradient
- Action Robust Reinforcement Learning and Applications in Continuous Control
- Multi-Agent Adversarial Inverse Reinforcement Learning
- The Approximate Duality Gap Technique: A Unified Theory of First-Order Methods
- Last-iterate convergence rates for min-max optimization