Towards Deep Symbolic Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) brings the power of deep neural networks to bear on the generic task of trial-and-error learning, and its effectiveness has been convincingly demonstrated on tasks such as Atari video games and the game of Go. However, contemporary DRL systems inherit a number of shortcomings from the current generation of deep learning techniques. For example, they require very large datasets to work effectively, entailing that they are slow to learn even when such datasets are available. Moreover, they lack the ability to reason on an abstract level, which makes it difficult to implement high-level cognitive functions such as transfer learning, analogical reasoning, and hypothesis-based reasoning. Finally, their operation is largely opaque to humans, rendering them unsuitable for domains in which verifiability is important. In this paper, we propose an end-to-end reinforcement learning architecture comprising a neural back end and a symbolic front end with the potential to overcome each of these shortcomings. As proof-of-concept, we present a preliminary implementation of the architecture and apply it to several variants of a simple video game. We show that the resulting system -- though just a prototype -- learns effectively, and, by acquiring a set of symbolic rules that are easily comprehensible to humans, dramatically outperforms a conventional, fully neural DRL system on a stochastic variant of the game.

References in corpus (1)

• Deep Learning in Neural Networks: An Overview

Cited by in corpus (8)

• A Brief Survey of Deep Reinforcement Learning
• A simple neural network module for relational reasoning
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems
• Injecting Prior Knowledge for Transfer Learning into Reinforcement Learning Algorithms using Logic Tensor Networks
• A method for the online construction of the set of states of a Markov Decision Process using Answer Set Programming
• Using Program Induction to Interpret Transition System Dynamics
• Explaining Transition Systems through Program Induction