Graph Constrained Reinforcement Learning for Natural Language Action Spaces
arXiv:2001.08837
Abstract
Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language. They are ideal environments for studying how to extend reinforcement learning agents to meet the challenges of natural language understanding, partial observability, and action generation in combinatorially-large text-based action spaces. We present KG-A2C, an agent that builds a dynamic knowledge graph while exploring and generates actions using a template-based action space. We contend that the dual uses of the knowledge graph to reason about game state and to constrain natural language generation are the keys to scalable exploration of combinatorially large natural language actions. Results across a wide variety of IF games show that KG-A2C outperforms current IF agents despite the exponential increase in action space size.
Accepted to ICLR 2020
References in corpus (1)
Cited by in corpus (8)
- How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds
- Playing Text-Based Games with Common Sense
- How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agents
- Learning Knowledge Graph-based World Models of Textual Environments
- Detecting and Adapting to Novelty in Games
- Modeling Worlds in Text
- Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning
- Generalization in Text-based Games via Hierarchical Reinforcement Learning