Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning
arXiv:1909.07543
Abstract
Continuous control tasks in reinforcement learning are important because they provide an important framework for learning in high-dimensional state spaces with deceptive rewards, where the agent can easily become trapped into suboptimal solutions. One way to avoid local optima is to use a population of agents to ensure coverage of the policy space, yet learning a population with the "best" coverage is still an open problem. In this work, we present a novel approach to population-based RL in continuous control that leverages properties of normalizing flows to perform attractive and repulsive operations between current members of the population and previously observed policies. Empirical results on the MuJoCo suite demonstrate a high performance gain for our algorithm compared to prior work, including Soft-Actor Critic (SAC).
References in corpus (12)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Trust Region Policy Optimization
- Variational Inference with Normalizing Flows
- #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
- Meta-Reinforcement Learning of Structured Exploration Strategies
- Stein Variational Policy Gradient
- Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
- Evolution-Guided Policy Gradient in Reinforcement Learning
- Boosting Trust Region Policy Optimization by Normalizing Flows Policy
- Randomized Value Functions via Multiplicative Normalizing Flows
- Bayesian Policy Gradients via Alpha Divergence Dropout Inference
- GAN Q-learning
Cited by in corpus (5)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Effective Diversity in Population Based Reinforcement Learning
- Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimization
- Harnessing Distribution Ratio Estimators for Learning Agents with Quality and Diversity
- Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces