93 citations · 429 across the 40 of their papers we have counts for
17 papers · 1 filter
Can Language Models Serve as Text-Based World Simulators?
Ruoyao Wang, Graham Todd, Ziang Xiao +4
Virtual environments play a key role in benchmarking advances in complex planning and decision-making tasks but are expensive and complicated to build by hand. Can current language…
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté +6
Large language models (LLMs) can be seen as atomic units of computation mapping sequences to a distribution over sequences. Thus, they can be seen as stochastic language layers in…
ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games
Ruoyao Wang, Graham Todd, Eric Yuan +3
In this work, we investigate the capacity of language models to generate explicit, interpretable, and interactive world models of scientific and common-sense reasoning tasks. We op…
Collecting Interactive Multi-modal Datasets for Grounded Language Understanding
Shrestha Mohanty, Negar Arabzadeh, Milagro Teruel +9
Human intelligence can remarkably adapt quickly to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by…
Behavior Cloned Transformers are Neurosymbolic Reasoners
Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté +1
In this work, we explore techniques for augmenting interactive agents with information from symbolic modules, much like humans use tools like calculators and GPS systems to assist…
TextWorldExpress: Simulating Text Games at One Million Steps Per Second
Peter A. Jansen, Marc-Alexandre Côté
Text-based games offer a challenging test bed to evaluate virtual agents at language understanding, multi-step problem-solving, and common-sense reasoning. However, speed is a majo…
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Julia Kiseleva, Alexey Skrynnik, Artem Zholus +14
Human intelligence has the remarkable ability to adapt to new tasks and environments quickly. Starting from a very young age, humans acquire new skills and learn how to solve new t…
One-Shot Learning from a Demonstration with Hierarchical Latent Language
Nathaniel Weir, Xingdi Yuan, Marc-Alexandre Côté +5
Humans have the capability, aided by the expressive compositionality of their language, to learn quickly by demonstration. They are able to describe unseen task-performing procedur…
ScienceWorld: Is your Agent Smarter than a 5th Grader?
Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté +1
We present ScienceWorld, a benchmark to test agents' scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curri…
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté +3
Given a simple request like Put a washed apple in the kitchen fridge, humans can reason in purely abstract terms by imagining action sequences and scoring their likelihood of succe…
Learning Dynamic Belief Graphs to Generalize on Text-Based Games
Ashutosh Adhikari, Xingdi Yuan, Marc-Alexandre Côté +7
Playing text-based games requires skills in processing natural language and sequential decision making. Achieving human-level performance on text-based games remains an open challe…
Building Dynamic Knowledge Graphs from Text-based Games
Mikuláš Zelinka, Xingdi Yuan, Marc-Alexandre Côté +2
We are interested in learning how to update Knowledge Graphs (KG) from text. In this preliminary work, we propose a novel Sequence-to-Sequence (Seq2Seq) architecture to generate el…
Interactive Language Learning by Question Answering
Xingdi Yuan, Marc-Alexandre Cote, Jie Fu +4
Humans observe and interact with the world to acquire knowledge. However, most existing machine reading comprehension (MRC) tasks miss the interactive, information-seeking componen…
Interactive Machine Comprehension with Information Seeking Agents
Xingdi Yuan, Jie Fu, Marc-Alexandre Cote +3
Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). We argue t…
Lessons learned in multilingual grounded language learning
Ákos Kádár, Desmond Elliott, Marc-Alexandre Côté +2
Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language. Here, we investigate in detail which conditions aff…
Revisiting the Hierarchical Multiscale LSTM
Ákos Kádár, Marc-Alexandre Côté, Grzegorz Chrupała +1
Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art language model that learns interpretable structure from character-level input. Such models can provide fert…
Counting to Explore and Generalize in Text-based Games
Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni +4
We propose a recurrent RL agent with an episodic exploration mechanism that helps discovering good policies in text-based game environments. We show promising results on a set of g…