The NetHack Learning Environment

Abstract

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both. Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open source at https://github.com/facebookresearch/nle.

arXiv comment28 pages. Accepted at NeurIPS 2020

References in corpus (21)

• A Brief Survey of Deep Reinforcement Learning
• StarCraft II: A New Challenge for Reinforcement Learning
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
• Count-Based Exploration with Neural Density Models
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
• Go-Explore: a New Approach for Hard-Exploration Problems
• Leveraging Procedural Generation to Benchmark Reinforcement Learning
• Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
• Language as an Abstraction for Hierarchical Deep Reinforcement Learning
• TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games
• Gotta Learn Fast: A New Benchmark for Generalization in RL
• Learning Montezuma's Revenge from a Single Demonstration
• BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
• RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
• TorchBeast: A PyTorch Platform for Distributed RL
• Value Propagation Networks
• The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors
• Rotation, Translation, and Cropping for Zero-Shot Generalization
• CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant
• Dungeon Crawl Stone Soup as an Evaluation Domain for Artificial Intelligence

Cited by in corpus (16)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• Learning with AMIGo: Adversarially Motivated Intrinsic Goals
• ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
• BeBold: Exploration Beyond the Boundary of Explored Regions
• Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning
• Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
• CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
• Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments
• Decoupling Value and Policy for Generalization in Reinforcement Learning
• Prioritized Level Replay
• CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning
• SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark
• Benchmarking the Spectrum of Agent Capabilities
• Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games