Implicit Policy for Reinforcement Learning

Abstract

We introduce Implicit Policy, a general class of expressive policies that can flexibly represent complex action distributions in reinforcement learning, with efficient algorithms to compute entropy regularized policy gradients. We empirically show that, despite its simplicity in implementation, entropy regularization combined with a rich policy class can attain desirable properties displayed under maximum entropy reinforcement learning framework, such as robustness and multi-modality.

References in corpus (6)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• NICE: Non-linear Independent Components Estimation
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Simple random search provides a competitive approach to reinforcement learning
• Latent Space Policies for Hierarchical Reinforcement Learning

Cited by in corpus (4)

• Boosting Trust Region Policy Optimization by Normalizing Flows Policy
• Discretizing Continuous Action Space for On-Policy Optimization
• Implicit Distributional Reinforcement Learning
• Efficient Wasserstein Natural Gradients for Reinforcement Learning