Control of Memory, Active Perception, and Action in Minecraft

Abstract

In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world). We then use these tasks to systematically compare and contrast existing deep reinforcement learning (DRL) architectures with our new memory-based DRL architectures. These tasks are designed to emphasize, in a controllable manner, issues that pose challenges for RL methods including partial observability (due to first-person visual observations), delayed rewards, high-dimensional visual observations, and the need to use active perception in a correct manner so as to perform well in the tasks. While these tasks are conceptually simple to describe, by virtue of having all of these challenges simultaneously they are difficult for current DRL architectures. Additionally, we evaluate the generalization performance of the architectures on environments not used during training. The experimental results show that our new architectures generalize to unseen environments better than existing DRL architectures.

ICML 2016

References in corpus (8)

• Deep Learning in Neural Networks: An Overview
• Continuous control with deep reinforcement learning
• Trust Region Policy Optimization
• End-To-End Memory Networks
• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Neural Turing Machines
• MazeBase: A Sandbox for Learning from Games

Cited by in corpus (78)

• Unity: A General Platform for Intelligent Agents
• Deep Reinforcement Learning: An Overview
• On Evaluation of Embodied Navigation Agents
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• Sample Efficient Actor-Critic with Experience Replay
• Differentiable MPC for End-to-end Planning and Control
• Transfer Learning in Deep Reinforcement Learning: A Survey
• Unsupervised Predictive Memory in a Goal-Directed Agent
• Neural SLAM: Learning to Explore with External Memory
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Energy Management Based on Multi-Agent Deep Reinforcement Learning for A Multi-Energy Industrial Park
• Neural Episodic Control
• CHALET: Cornell House Agent Learning Environment
• Chasing Ghosts: Instruction Following as Bayesian State Tracking
• Semi-parametric Topological Memory for Navigation
• The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors
• Neural Certificates for Safe Control Policies
• Value Prediction Network
• Benchmarking Classic and Learned Navigation in Complex 3D Environments
• Generalizing Skills with Semi-Supervised Reinforcement Learning
• Entity Abstraction in Visual Model-Based Reinforcement Learning
• Deep Reinforcement Learning from Policy-Dependent Human Feedback
• Improving the Gating Mechanism of Recurrent Neural Networks
• Multi-focus Attention Network for Efficient Deep Reinforcement Learning
• Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds
• A Critical Investigation of Deep Reinforcement Learning for Navigation
• Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships
• CraftAssist: A Framework for Dialogue-enabled Interactive Agents
• Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
• Learning to Navigate in Indoor Environments: from Memorizing to Reasoning
• Deep Reinforcement Learning with Model Learning and Monte Carlo Tree Search in Minecraft
• Synthetic Returns for Long-Term Credit Assignment
• A Framework for Searching for General Artificial Intelligence
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
• Why Build an Assistant in Minecraft?
• Learning to Visually Navigate in Photorealistic Environments Without any Supervision
• Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goals
• AndroidEnv: A Reinforcement Learning Platform for Android
• Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learning
• CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant
• Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
• Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition
• Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
• Complementary reinforcement learning towards explainable agents
• SGoLAM: Simultaneous Goal Localization and Mapping for Multi-Object Goal Navigation
• Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings
• Maximizing Information Gain in Partially Observable Environments via Prediction Reward
• Optimize Neural Fictitious Self-Play in Regret Minimization Thinking
• Scene-Intuitive Agent for Remote Embodied Visual Grounding
• The act of remembering: a study in partially observable reinforcement learning
• Cross-View Policy Learning for Street Navigation
• Exploration for Multi-task Reinforcement Learning with Deep Generative Models
• Understanding and Controlling Memory in Recurrent Neural Networks
• MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation
• Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
• Modeling Latent Attention Within Neural Networks
• Deep Reinforcement Learning From Raw Pixels in Doom
• Towards robust and domain agnostic reinforcement learning competitions
• Deep Reinforcement Learning for Smart Grid Protection Against Coordinated Multistage Transmission Line Attacks
• Stabilizing Transformer-Based Action Sequence Generation For Q-Learning
• Working Memory Graphs
• Critic PI2: Master Continuous Planning via Policy Improvement with Path Integrals and Deep Actor-Critic Reinforcement Learning
• Simultaneous Navigation and Construction Benchmarking Environments
• Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control with Scarce Data and Side Information
• Leveraging Motion Priors in Videos for Improving Human Segmentation
• Planning from Images with Deep Latent Gaussian Process Dynamics
• How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem
• Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF
• Episodic Memory for Learning Subjective-Timescale Models
• Rapid Task-Solving in Novel Environments
• A2: Extracting Cyclic Switchings from DOB-nets for Rejecting Excessive Disturbances
• Building Intelligent Autonomous Navigation Agents
• Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain
• Evolving Indoor Navigational Strategies Using Gated Recurrent Units In NEAT
• Mature GAIL: Imitation Learning for Low-level and High-dimensional Input using Global Encoder and Cost Transformation
• Graph Convolutional Memory using Topological Priors