Boosting Trust Region Policy Optimization by Normalizing Flows Policy
arXiv:1809.10326
Abstract
We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps avoid bad local optima. Through extensive comparisons, we show that the normalizing flows policy significantly improves upon baseline architectures especially on high-dimensional tasks with complex dynamics.
References in corpus (13)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Trust Region Policy Optimization
- NICE: Non-linear Independent Components Estimation
- Variational Inference with Normalizing Flows
- Benchmarking Deep Reinforcement Learning for Continuous Control
- Density estimation using Real NVP
- Optimizing Neural Networks with Kronecker-factored Approximate Curvature
- Reinforcement Learning with Deep Energy-Based Policies
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- Simple random search provides a competitive approach to reinforcement learning
- Latent Space Policies for Hierarchical Reinforcement Learning
- Implicit Policy for Reinforcement Learning
Cited by in corpus (12)
- Discrete and Continuous Action Representation for Practical RL in Video Games
- Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies
- Leveraging exploration in off-policy algorithms via normalizing flows
- Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
- Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning
- Iterative Amortized Policy Optimization
- A unified view of likelihood ratio and reparameterization gradients and an optimal importance sampling scheme
- Soft Actor-Critic With Integer Actions
- AI-Based and Mobility-Aware Energy Efficient Resource Allocation and Trajectory Design for NFV Enabled Aerial Networks
- PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Network
- Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow
- EnTRPO: Trust Region Policy Optimization Method with Entropy Regularization