Making Efficient Use of Demonstrations to Solve Hard Exploration Problems

Abstract

This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial conditions. We also introduce a suite of eight tasks that combine these three properties, and show that R2D3 can solve several of the tasks where other state of the art methods (both with and without demonstrations) fail to see even a single successful trajectory after tens of billions of steps of exploration.

References in corpus (14)

• Dueling Network Architectures for Deep Reinforcement Learning
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
• Distributed Prioritized Experience Replay
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Deep Q-learning from Demonstrations
• Benchmarking Model-Based Reinforcement Learning
• Go-Explore: a New Approach for Hard-Exploration Problems
• Learning human behaviors from motion capture by adversarial imitation
• Observe and Look Further: Achieving Consistent Performance on Atari
• Learning Montezuma's Revenge from a Single Demonstration
• Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
• One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL
• Reinforced Imitation in Heterogeneous Action Space

Cited by in corpus (12)

• Exploration in Deep Reinforcement Learning: A Survey
• A Minimalist Approach to Offline Reinforcement Learning
• Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Driving
• Offline Learning from Demonstrations and Unlabeled Experience
• Self-Imitation Advantage Learning
• Physically Embedded Planning Problems: New Challenges for Reinforcement Learning
• Adaptive Procedural Task Generation for Hard-Exploration Problems
• PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning
• Guided Exploration with Proximal Policy Optimization using a Single Demonstration
• Reinforcement Learning for Vision-based Object Manipulation with Non-parametric Policy and Action Primitives
• DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning
• Did I do that? Blame as a means to identify controlled effects in reinforcement learning