9 citations · 18 across the 7 of their papers we have counts for
7 papers
Decoupling Planning and Control for Instructable Agents
Zineng Tang, Kelsey R. Allen, Sjoerd van Steenkiste +2
Recent work shows that pre-trained, instruction-tuned vision-language models (VLMs) perform well at mapping from instructions and observations to high-level plans, but struggle to…
Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
Michael A. Lepori, Jennifer Hu, Ishita Dasgupta +3
Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able t…
ReMI: A Dataset for Reasoning with Multiple Images
Mehran Kazemi, Nishanth Dikkala, Ankit Anand +8
With the continuous advancement of large language models (LLMs), it is essential to create new benchmarks to effectively evaluate their expanding capabilities and identify areas fo…
Scaling Instructable Agents Across Many Simulated Worlds
SIMA Team, Maria Abi Raad, Arun Ahuja +91
Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI. Accomplishing this goal requires lear…
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Soroush Nasiriany, Fei Xia, Wenhao Yu +20
Vision language models (VLMs) have shown impressive capabilities across a variety of tasks, from logical reasoning to visual understanding. This opens the door to richer interactio…
A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models
Tiwalayo Eisape, MH Tessler, Ishita Dasgupta +3
A central component of rational behavior is logical inference: the process of determining which conclusions follow from a set of premises. Psychologists have documented several way…
The Impact of Depth on Compositional Generalization in Transformer Language Models
Jackson Petty, Sjoerd van Steenkiste, Ishita Dasgupta +3
To process novel sentences, language models (LMs) must generalize compositionally -- combine familiar elements in new ways. What aspects of a model's structure promote compositiona…