42 citations · 80 across the 15 of their papers we have counts for
17 papers
AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use
Junzhi Chen, Harsh Trivedi, Jane Pan +4
Tool-use agents that address day-to-day digital tasks such as ordering groceries must not only operate applications, but also interact with the user, e.g., to ask clarification que…
Democratizing the medieval English legal tradition
Michael Zhang, Elise Wang, Charlotte Whatley +2
The record of the beginning of the most widespread legal system in the world is contained in millions of pages of handwritten text. Most of the records of the first centuries of th…
User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal
Yuhan Liu, Michael J. Q. Zhang, Eunsol Choi
Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback. Asking for direct user feedback can be disruptive; thus,…
Improving LLM-as-a-Judge Inference with the Judgment Distribution
Victor Wang, Michael J. Q. Zhang, Eunsol Choi
Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) has become a standard practice applicable to many tasks. A judgment is often extrac…
Diverging Preferences: When do Annotators Disagree and do Models Know?
Michael JQ Zhang, Zhilin Wang, Jena D. Hwang +6
We examine diverging preferences in human-labeled preference datasets. We develop a taxonomy of disagreement sources spanning ten categories across four high-level classes and find…
Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions
Michael J. Q. Zhang, W. Bradley Knox, Eunsol Choi
Large language models (LLMs) must often respond to highly ambiguous user requests. In such cases, the LLM's best response may be to ask a clarifying question to elicit more informa…
Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs
Michael J. Q. Zhang, Eunsol Choi
Resolving ambiguities through interaction is a hallmark of natural language, and modeling this behavior is a core challenge in crafting AI assistants. In this work, we study such b…
Propagating Knowledge Updates to LMs Through Distillation
Shankar Padmanabhan, Yasumasa Onoe, Michael J. Q. Zhang +2
Modern language models have the capacity to store and use immense amounts of knowledge about real-world entities, but it remains unclear how to update such knowledge stored in mode…
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge
Yasumasa Onoe, Michael J. Q. Zhang, Shankar Padmanabhan +2
Pre-trained language models (LMs) are used for knowledge intensive tasks like question answering, but their knowledge gets continuously outdated as the world changes. Prior work ha…
Mitigating Temporal Misalignment by Discarding Outdated Facts
Michael J. Q. Zhang, Eunsol Choi
While large language models are able to retain vast amounts of world knowledge seen during pretraining, such knowledge is prone to going out of date and is nontrivial to update. Fu…
Selectively Answering Ambiguous Questions
Jeremy R. Cole, Michael J. Q. Zhang, Daniel Gillick +3
Trustworthy language models should abstain from answering questions when they do not know the answer. However, the answer to a question can be unknown for a variety of reasons. Pri…
DIFFQG: Generating Questions to Summarize Factual Changes
Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos +3
Identifying the difference between two versions of the same article is useful to update knowledge bases and to understand how articles evolve. Paired texts occur naturally in diver…
Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence
Hung-Ting Chen, Michael J. Q. Zhang, Eunsol Choi
Question answering models can use rich knowledge sources -- up to one hundred retrieved passages and parametric knowledge in the large-scale language model (LM). Prior work assumes…
Entity Cloze By Date: What LMs Know About Unseen Entities
Yasumasa Onoe, Michael J. Q. Zhang, Eunsol Choi +1
Language models (LMs) are typically trained once on a large-scale corpus and used for years without being updated. However, in a dynamic world, new entities constantly arise. We pr…
SituatedQA: Incorporating Extra-Linguistic Contexts into QA
Michael J. Q. Zhang, Eunsol Choi
Answers to the same question may change depending on the extra-linguistic contexts (when and where the question was asked). To study this challenge, we introduce SituatedQA, an ope…
CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge
Yasumasa Onoe, Michael J. Q. Zhang, Eunsol Choi +1
Most benchmark datasets targeting commonsense reasoning focus on everyday scenarios: physical knowledge like knowing that you could fill a cup under a waterfall [Talmor et al., 201…
Deep Weighted Averaging Classifiers
Dallas Card, Michael Zhang, Noah A. Smith
Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, c…