The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors

Abstract

Though deep reinforcement learning has led to breakthroughs in many difficult domains, these successes have required an ever-increasing number of samples. As state-of-the-art reinforcement learning (RL) systems require an exponentially increasing number of samples, their development is restricted to a continually shrinking segment of the AI community. Likewise, many of these systems cannot be applied to real-world problems, where environment samples are expensive. Resolution of these limitations requires new, sample-efficient methods. To facilitate research in this direction, we introduce the MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors. The primary goal of the competition is to foster the development of algorithms which can efficiently leverage human demonstrations to drastically reduce the number of samples needed to solve complex, hierarchical, and sparse environments. To that end, we introduce: (1) the Minecraft ObtainDiamond task, a sequential decision making environment requiring long-term planning, hierarchical control, and efficient exploration methods; and (2) the MineRL-v0 dataset, a large-scale collection of over 60 million state-action pairs of human demonstrations that can be resimulated into embodied trajectories with arbitrary modifications to game state and visuals. Participants will compete to develop systems which solve the ObtainDiamond task with a limited number of samples from the environment simulator, Malmo. The competition is structured into two rounds in which competitors are provided several paired versions of the dataset and environment with different game textures. At the end of each round, competitors will submit containerized versions of their learning algorithms and they will then be trained/evaluated from scratch on a hold-out dataset-environment pair for a total of 4-days on a prespecified hardware platform.

arXiv commentaccepted at NeurIPS 2019, 28 pages

References in corpus (17)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• End to End Learning for Self-Driving Cars
• Dueling Network Architectures for Deep Reinforcement Learning
• Dota 2 with Large Scale Deep Reinforcement Learning
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Deep Q-learning from Demonstrations
• One-Shot Visual Imitation Learning via Meta-Learning
• Control of Memory, Active Perception, and Action in Minecraft
• Reinforcement Learning from Imperfect Demonstrations
• Gotta Learn Fast: A New Benchmark for Generalization in RL
• ChainerRL: A Deep Reinforcement Learning Library
• Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning
• The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
• SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
• The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
• Guaranteeing Reproducibility in Deep Learning Competitions

Cited by in corpus (19)

• Dota 2 with Large Scale Deep Reinforcement Learning
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• 10 Years of the PCG workshop: Past and Future Trends
• CraftAssist: A Framework for Dialogue-enabled Interactive Agents
• Learning to Play No-Press Diplomacy with Best Response Policy Iteration
• Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Games
• Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills
• Reset-Free Lifelong Learning with Skill-Space Planning
• CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
• Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learning
• The MineRL BASALT Competition on Learning from Human Feedback
• Playing Minecraft with Behavioural Cloning
• Approximate Model-Based Shielding for Safe Reinforcement Learning
• Adaptive Agent Architecture for Real-time Human-Agent Teaming
• Towards robust and domain agnostic reinforcement learning competitions
• SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competition
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation
• Unsupervised Skill-Discovery and Skill-Learning in Minecraft