A Deep Hierarchical Approach to Lifelong Learning in Minecraft

Abstract

We propose a lifelong learning system that has the ability to reuse and transfer knowledge from one task to another while efficiently retaining the previously learned knowledge-base. Knowledge is transferred by learning reusable skills to solve tasks in Minecraft, a popular video game which is an unsolved and high-dimensional lifelong learning problem. These reusable skills, which we refer to as Deep Skill Networks, are then incorporated into our novel Hierarchical Deep Reinforcement Learning Network (H-DRLN) architecture using two techniques: (1) a deep skill array and (2) skill distillation, our novel variation of policy distillation (Rusu et. al. 2015) for learning skills. Skill distillation enables the HDRLN to efficiently retain knowledge and therefore scale in lifelong learning, by accumulating knowledge and encapsulating multiple reusable skills into a single distilled network. The H-DRLN exhibits superior performance and lower learning sample complexity compared to the regular Deep Q Network (Mnih et. al. 2015) in sub-domains of Minecraft.

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• Asynchronous Methods for Deep Reinforcement Learning
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Hierarchical Solution of Markov Decision Processes using Macro-actions
• Strategic Attentive Writer for Learning Macro-Actions
• Adaptive Skills, Adaptive Partitions (ASAP)
• Iterative Hierarchical Optimization for Misspecified Problems (IHOMP)

Cited by in corpus (50)

• Deep Reinforcement Learning: An Overview
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• A Survey of Deep Reinforcement Learning in Video Games
• Deep Learning in Mobile and Wireless Networking: A Survey
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms
• Strategic Attentive Writer for Learning Macro-Actions
• Latent Space Policies for Hierarchical Reinforcement Learning
• Some Considerations on Learning to Explore via Meta-Reinforcement Learning
• An empirical investigation of the challenges of real-world reinforcement learning
• Robust Reinforcement Learning for Continuous Control with Model Misspecification
• Unicorn: Continual Learning with a Universal, Off-policy Agent
• Hierarchical Decision Making by Generating and Following Natural Language Instructions
• Deep Reinforcement Learning from Policy-Dependent Human Feedback
• AI Research Considerations for Human Existential Safety (ARCHES)
• Active Long Term Memory Networks
• Aggregating E-commerce Search Results from Heterogeneous Sources via Hierarchical Reinforcement Learning
• Deep Reinforcement Learning for Dexterous Manipulation with Concept Networks
• Transferring Autonomous Driving Knowledge on Simulated and Real Intersections
• Visualizing Dynamics: from t-SNE to SEMI-MDPs
• Analyzing Knowledge Transfer in Deep Q-Networks for Autonomously Handling Multiple Intersections
• Iterative Hierarchical Optimization for Misspecified Problems (IHOMP)
• Learning latent representations across multiple data domains using Lifelong VAEGAN
• Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learning
• CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant
• Neural Arithmetic Expression Calculator
• Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition
• Environments for Lifelong Reinforcement Learning
• The Effects of Memory Replay in Reinforcement Learning
• Representative Task Self-selection for Flexible Clustered Lifelong Learning
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Learning to Play General Video-Games via an Object Embedding Network
• DRL: Deep Reinforcement Learning for Intelligent Robot Control -- Concept, Literature, and Future
• Program Synthesis Guided Reinforcement Learning for Partially Observed Environments
• Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
• Deep Reinforcement Learning Discovers Internal Models
• Deep Reinforcement Learning From Raw Pixels in Doom
• Walking with MIND: Mental Imagery eNhanceD Embodied QA
• Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
• Feudal Steering: Hierarchical Learning for Steering Angle Prediction
• Zero-shot task adaptation by homoiconic meta-mapping
• Growing a Brain: Fine-Tuning by Increasing Model Capacity
• Hierarchical Reinforcement Learning with Deep Nested Agents
• Variable-Shot Adaptation for Online Meta-Learning
• Planning in Hierarchical Reinforcement Learning: Guarantees for Using Local Policies
• Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning
• Lifetime policy reuse and the importance of task capacity
• Hierarchical Reinforcement Learning: Approximating Optimal Discounted TSP Using Local Policies
• Reducing the Deployment-Time Inference Control Costs of Deep Reinforcement Learning Agents via an Asymmetric Architecture
• HMRL: Hyper-Meta Learning for Sparse Reward Reinforcement Learning Problem