What can you do with a rock? Affordance extraction via word embeddings

Abstract

Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance detection is particularly helpful in domains with large action spaces, allowing the agent to prune its search space by avoiding futile behaviors. This paper presents a method for affordance extraction via word embeddings trained on a Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance most of the time. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent's action selections begin to resemble those a human would choose.

7 pages, 7 figures, 2 algorithms, data runs were performed using the Autoplay learning environment for interactive fiction

References in corpus (2)

• Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
• Detecting Affordances by Visuomotor Simulation

Cited by in corpus (9)

• Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning
• Cultural Cartography with Word Embeddings
• Counting to Explore and Generalize in Text-based Games
• WordCraft: An Environment for Benchmarking Commonsense Agents
• Comprehensible Context-driven Text Game Playing
• Keep CALM and Explore: Language Models for Action Generation in Text-based Games
• Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games
• Temporally Abstract Partial Models
• Sentiment Analysis for Reinforcement Learning