Grounded Language Learning in a Simulated 3D World
arXiv:1706.06551
Abstract
We are increasingly surrounded by artificially intelligent technology that takes decisions and executes actions on our behalf. This creates a pressing need for general means to communicate with, instruct and guide artificial agents, with human language the most compelling means for such communication. To achieve this in a scalable fashion, agents must be able to relate language to the world and to actions; that is, their understanding of language must be grounded and embodied. However, learning grounded language is a notoriously challenging problem in artificial intelligence research. Here we present an agent that learns to interpret language in a simulated 3D environment where it is rewarded for the successful execution of written instructions. Trained via a combination of reinforcement and unsupervised learning, and beginning with minimal prior knowledge, the agent learns to relate linguistic symbols to emergent perceptual representations of its physical surroundings and to pertinent sequences of actions. The agent's comprehension of language extends beyond its prior experience, enabling it to apply familiar language to unfamiliar situations and to interpret entirely novel instructions. Moreover, the speed with which this agent learns new words increases as its semantic knowledge grows. This facility for generalising and bootstrapping semantic knowledge indicates the potential of the present approach for reconciling ambiguous natural language with the complexity of the physical world.
16 pages, 8 figures
References in corpus (3)
Cited by in corpus (84)
- Object Goal Navigation using Goal-Oriented Semantic Exploration
- Unsupervised Predictive Memory in a Goal-Directed Agent
- Symbol Emergence in Cognitive Developmental Systems: a Survey
- TextWorld: A Learning Environment for Text-based Games
- HoME: a Household Multimodal Environment
- CHALET: Cornell House Agent Learning Environment
- Learning to Generalize from Sparse and Underspecified Rewards
- Meta reinforcement learning as task inference
- Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents
- Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning
- Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory
- Learning to Navigate in Cities Without a Map
- Imitating Interactive Intelligence
- FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learning
- Grounded Language Learning Fast and Slow
- Unicorn: Continual Learning with a Universal, Off-policy Agent
- Hierarchical Decision Making by Generating and Following Natural Language Instructions
- Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text
- Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
- Emergence of Communication in an Interactive World with Consistent Speakers
- Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
- Universal Successor Features Approximators
- Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents
- Understanding Early Word Learning in Situated Artificial Agents
- Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View
- Multi-Task Reinforcement Learning with Context-based Representations
- The Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks Simultaneously
- LTL2Action: Generalizing LTL Instructions for Multi-Task RL
- The Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents
- Multimodal Grounding for Language Processing
- Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
- Situational Fusion of Visual Representation for Visual Navigation
- Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning
- RTFM: Generalising to Novel Environment Dynamics via Reading
- Transforming task representations to perform novel tasks
- Grounding Language to Autonomously-Acquired Skills via Goal Generation
- Embodied Question Answering in Photorealistic Environments with Point Cloud Perception
- A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution
- Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation
- lamBERT: Language and Action Learning Using Multimodal BERT
- Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
- Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision
- ELLA: Exploration through Learned Language Abstraction
- Language-Conditioned Goal Generation: a New Approach to Language Grounding for RL
- NLPGym -- A toolkit for evaluating RL agents on Natural Language Processing Tasks
- Unsupervised Domain Adaptation for Visual Navigation
- Mastering emergent language: learning to guide in simulated navigation
- Following Instructions by Imagining and Reaching Visual Goals
- Probing Emergent Semantics in Predictive Agents via Question Answering
- Object-oriented Targets for Visual Navigation using Rich Semantic Representations
- Inductive Bias and Language Expressivity in Emergent Communication
- Symbolic Behaviour in Artificial Intelligence
- An Overview of Natural Language State Representation for Reinforcement Learning
- Interactive Learning from Activity Description
- Grounding Spatio-Temporal Language with Transformers
- Emergence of Numeric Concepts in Multi-Agent Autonomous Communication
- Language Conditioned Imitation Learning over Unstructured Data
- ArraMon: A Joint Navigation-Assembly Instruction Interpretation Task in Dynamic Environments
- Curriculum Q-Learning for Visual Vocabulary Acquisition
- Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent
- Pre-trained Word Embeddings for Goal-conditional Transfer Learning in Reinforcement Learning
- SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark
- On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator
- Learning Adaptive Language Interfaces through Decomposition
- Generative Language-Grounded Policy in Vision-and-Language Navigation with Bayes' Rule
- Exploiting Language Instructions for Interpretable and Compositional Reinforcement Learning
- SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents
- Systematic Generalization on gSCAN: What is Nearly Solved and What is Next?
- An Extensible Interactive Interface for Agent Design
- Automated curriculum generation for Policy Gradients from Demonstrations
- Zero-shot task adaptation by homoiconic meta-mapping
- Fast Task-Adaptation for Tasks Labeled Using Natural Language in Reinforcement Learning
- Fast Concept Mapping: The Emergence of Human Abilities in Artificial Neural Networks when Learning Embodied and Self-Supervised
- Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness
- Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining
- Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Control
- Feudal Reinforcement Learning by Reading Manuals
- Language coverage and generalization in RNN-based continuous sentence embeddings for interacting agents
- Resolving Referring Expressions in Images With Labeled Elements
- Compositional Networks Enable Systematic Generalization for Grounded Language Understanding
- A new dataset and model for learning to understand navigational instructions
- Toward Human-Level Artificial Intelligence
- Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe
- Building Intelligent Autonomous Navigation Agents