Acquiring Grounded Representations of Words with Situated Interactive Instruction

Abstract

We present an approach for acquiring grounded representations of words from mixed-initiative, situated interactions with a human instructor. The work focuses on the acquisition of diverse types of knowledge including perceptual, semantic, and procedural knowledge along with learning grounded meanings. Interactive learning allows the agent to control its learning by requesting instructions about unknown concepts, making learning efficient. Our approach has been instantiated in Soar and has been evaluated on a table-top robotic arm capable of manipulating small objects.

References in corpus (1)

• Grounded Semantic Composition for Visual Scenes

Cited by in corpus (8)

• A Review of 40 Years of Cognitive Architecture Research: Core Cognitive Abilities and Practical Applications
• A Computational Model for Situated Task Learning with Interactive Instruction
• GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments
• Semantic Robot Programming for Goal-Directed Manipulation in Cluttered Scenes
• Five dimensions of reasoning in the wild
• Towards an Indexical Model of Situated Language Comprehension for Cognitive Agents in Physical Worlds
• Conversational Multi-Hop Reasoning with Neural Commonsense Knowledge and Symbolic Logic Rules
• Learning a Policy for Opportunistic Active Learning