Learning Efficient and Effective Exploration Policies with Counterfactual Meta Policy

Abstract

A fundamental issue in reinforcement learning algorithms is the balance between exploration of the environment and exploitation of information already obtained by the agent. Especially, exploration has played a critical role for both efficiency and efficacy of the learning process. However, Existing works for exploration involve task-agnostic design, that is performing well in one environment, but be ill-suited to another. To the purpose of learning an effective and efficient exploration policy in an automated manner. We formalized a feasible metric for measuring the utility of exploration based on counterfactual ideology. Based on that, We proposed an end-to-end algorithm to learn exploration policy by meta-learning. We demonstrate that our method achieves good results compared to previous works in the high-dimensional control tasks in MuJoCo simulator.

References in corpus (17)

• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Playing Atari with Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Theoretical Models of Learning to Learn
• Noisy Networks for Exploration
• Counterfactual Multi-Agent Policy Gradients
• Parameter Space Noise for Exploration
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
• Exploration by Random Network Distillation
• Meta-Reinforcement Learning of Structured Exploration Strategies
• Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning
• Learning to Explore with Meta-Policy Gradient
• Meta-Learning for Contextual Bandit Exploration