Action Assembly: Sparse Imitation Learning for Text Based Games with Combinatorial Action Spaces

Abstract

We propose a computationally efficient algorithm that combines compressed sensing with imitation learning to solve text-based games with combinatorial action spaces. Specifically, we introduce a new compressed sensing algorithm, named IK-OMP, which can be seen as an extension to the Orthogonal Matching Pursuit (OMP). We incorporate IK-OMP into a supervised imitation learning setting and show that the combined approach (Sparse Imitation Learning, Sparse-IL) solves the entire text-based game of Zork1 with an action space of approximately 10 million actions given both perfect and noisy demonstrations.

arXiv commentUnder review at IJCAI 2020

References in corpus (9)

• Efficient Estimation of Word Representations in Vector Space
• Sequence Level Training with Recurrent Neural Networks
• Spatial Compressive Sensing for MIMO Radar
• An Actor-Critic Algorithm for Sequence Prediction
• Language Understanding for Text-based Games Using Deep Reinforcement Learning
• TextWorld: A Learning Environment for Text-based Games
• Counting to Explore and Generalize in Text-based Games
• Using reinforcement learning to learn how to play text-based games
• Towards Solving Text-based Games by Producing Adaptive Action Spaces

Cited by in corpus (5)

• An empirical investigation of the challenges of real-world reinforcement learning
• How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agents
• Language is Power: Representing States Using Natural Language in Reinforcement Learning
• Zero-Shot Learning of Text Adventure Games with Sentence-Level Semantics
• Learning to Generalize for Sequential Decision Making