The Thing That We Tried Didn't Work Very Well : Deictic Representation in Reinforcement Learning

Abstract

Most reinforcement learning methods operate on propositional representations of the world state. Such representations are often intractably large and generalize poorly. Using a deictic representation is believed to be a viable alternative: they promise generalization while allowing the use of existing reinforcement-learning methods. Yet, there are few experiments on learning with deictic representations reported in the literature. In this paper we explore the effectiveness of two forms of deictic representation and a naïve propositional representation in a simple blocks-world domain. We find, empirically, that the deictic representations actually worsen learning performance. We conclude with a discussion of possible causes of these results and strategies for more effective learning in domains with objects.

arXiv commentAppears in Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI2002)

References in corpus (1)

• Learning Finite-State Controllers for Partially Observable Environments

Cited by in corpus (4)

• Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering
• Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learning
• Learning Portable Representations for High-Level Planning
• An Overview of Natural Language State Representation for Reinforcement Learning