Some Considerations on Learning to Explore via Meta-Reinforcement Learning

Abstract

We consider the problem of exploration in meta reinforcement learning. Two new meta reinforcement learning algorithms are suggested: E-MAML and E-${\text{RL}}^2$. Results are presented on a novel environment we call `Krazy World' and a set of maze environments. We show E-MAML and E-${\text{RL}}^2$ deliver better performance on tasks where exploration is important.

References in corpus (8)

• Continuous control with deep reinforcement learning
• Asynchronous Methods for Deep Reinforcement Learning
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• VIME: Variational Information Maximizing Exploration
• Count-Based Exploration with Neural Density Models
• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
• A Deep Hierarchical Approach to Lifelong Learning in Minecraft
• On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models

Cited by in corpus (28)

• Meta-Reinforcement Learning of Structured Exploration Strategies
• Evolved Policy Gradients
• VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
• Unsupervised Meta-Learning for Reinforcement Learning
• A Comprehensive Overview and Survey of Recent Advances in Meta-Learning
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
• Meta-learning curiosity algorithms
• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
• Concurrent Meta Reinforcement Learning
• Context Meta-Reinforcement Learning via Neuromodulation
• Meta Reinforcement Learning with Task Embedding and Shared Policy
• Offline Meta Learning of Exploration
• Curriculum in Gradient-Based Meta-Reinforcement Learning
• A Survey of Exploration Methods in Reinforcement Learning
• Meta-Model-Based Meta-Policy Optimization
• Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning
• Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning
• Hindsight Foresight Relabeling for Meta-Reinforcement Learning
• Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learning
• A Brief Look at Generalization in Visual Meta-Reinforcement Learning
• OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation
• MAME : Model-Agnostic Meta-Exploration
• Zero-shot task adaptation by homoiconic meta-mapping
• Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation
• Bootstrapped Meta-Learning
• Behaviour-conditioned policies for cooperative reinforcement learning tasks
• On the Practical Consistency of Meta-Reinforcement Learning Algorithms