Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following
arXiv:1907.09671
Abstract
We consider the problem of learning to map from natural language instructions to state transitions (actions) in a data-efficient manner. Our method takes inspiration from the idea that it should be easier to ground language to concepts that have already been formed through pre-linguistic observation. We augment a baseline instruction-following learner with an initial environment-learning phase that uses observations of language-free state transitions to induce a suitable latent representation of actions before processing the instruction-following training data. We show that mapping to pre-learned representations substantially improves performance over systems whose representations are learned from limited instructional data alone.
arXiv commentACL 2019
References in corpus (5)
- Categorical Reparameterization with Gumbel-Softmax
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences