51 citations · 126 across the 16 of their papers we have counts for
16 papers · 1 filter
UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations
Wenting Zhao, Justin T Chiu, Jena D. Hwang +6
Language technologies that accurately model the dynamics of events must perform commonsense reasoning. Existing work evaluating commonsense reasoning focuses on making inferences a…
What's In My Big Data?
Yanai Elazar, Akshita Bhagia, Ian Magnusson +10
Large text corpora are the backbone of language models. However, we have a limited understanding of the content of these corpora, including general statistics, quality, social fact…
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
Melanie Sclar, Yejin Choi, Yulia Tsvetkov +1
As large language models (LLMs) are adopted as a fundamental component of language technologies, it is crucial to accurately characterize their performance. Because choices in prom…
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training
Zeqiu Wu, Yushi Hu, Weijia Shi +6
Language models (LMs) often exhibit undesirable text generation behaviors, including generating false, toxic, or irrelevant outputs. Reinforcement learning from human feedback (RLH…
Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker
Melanie Sclar, Sachin Kumar, Peter West +3
Theory of Mind (ToM)$\unicode{x2014}$the ability to reason about the mental states of other people$\unicode{x2014}$is a key element of our social intelligence. Yet, despite their e…
We're Afraid Language Models Aren't Modeling Ambiguity
Alisa Liu, Zhaofeng Wu, Julian Michael +6
Ambiguity is an intrinsic feature of natural language. Managing ambiguity is a key part of human language understanding, allowing us to anticipate misunderstanding as communicators…
Continual Learning for Instruction Following from Realtime Feedback
Alane Suhr, Yoav Artzi
We propose and deploy an approach to continually train an instruction-following agent from feedback provided by users during collaborative interactions. During interaction, human u…
Abstract Visual Reasoning with Tangram Shapes
Anya Ji, Noriyuki Kojima, Noah Rush +4
We introduce KiloGram, a resource for studying abstract visual reasoning in humans and machines. Drawing on the history of tangram puzzles as stimuli in cognitive science, we build…
Analysis of Language Change in Collaborative Instruction Following
Anna Effenberger, Eva Yan, Rhia Singh +2
We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase their expertise. Prio…
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior
Noriyuki Kojima, Alane Suhr, Yoav Artzi
We study continual learning for natural language instruction generation, by observing human users' instruction execution. We focus on a collaborative scenario, where the system bot…
Executing Instructions in Situated Collaborative Interactions
Alane Suhr, Claudia Yan, Charlotte Schluger +5
We study a collaborative scenario where a user not only instructs a system to complete tasks, but also acts alongside it. This allows the user to adapt to the system abilities by c…
NLVR2 Visual Bias Analysis
Alane Suhr, Yoav Artzi
NLVR2 (Suhr et al., 2019) was designed to be robust for language bias through a data collection process that resulted in each natural language sentence appearing with both true and…
A Corpus for Reasoning About Natural Language Grounded in Photographs
Alane Suhr, Stephanie Zhou, Ally Zhang +3
We introduce a new dataset for joint reasoning about natural language and images, with a focus on semantic diversity, compositionality, and visual reasoning challenges. The data co…
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation
Alane Suhr, Yoav Artzi
We propose a learning approach for mapping context-dependent sequential instructions to actions. We address the problem of discourse and state dependencies with an attention-based…
Learning to Map Context-Dependent Sentences to Executable Formal Queries
Alane Suhr, Srinivasan Iyer, Yoav Artzi
We propose a context-dependent model to map utterances within an interaction to executable formal queries. To incorporate interaction history, the model maintains an interaction-le…
Visual Reasoning with Natural Language
Stephanie Zhou, Alane Suhr, Yoav Artzi
Natural language provides a widely accessible and expressive interface for robotic agents. To understand language in complex environments, agents must reason about the full range o…