5 papers
Democratizing the medieval English legal tradition
Michael Zhang, Elise Wang, Charlotte Whatley +2
The record of the beginning of the most widespread legal system in the world is contained in millions of pages of handwritten text. Most of the records of the first centuries of th…
Diverging Preferences: When do Annotators Disagree and do Models Know?
Michael JQ Zhang, Zhilin Wang, Jena D. Hwang +6
We examine diverging preferences in human-labeled preference datasets. We develop a taxonomy of disagreement sources spanning ten categories across four high-level classes and find…
User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal
Yuhan Liu, Michael J. Q. Zhang, Eunsol Choi
Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback. Asking for direct user feedback can be disruptive; thus,…
Improving LLM-as-a-Judge Inference with the Judgment Distribution
Victor Wang, Michael J. Q. Zhang, Eunsol Choi
Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) has become a standard practice applicable to many tasks. A judgment is often extrac…
Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions
Michael J. Q. Zhang, W. Bradley Knox, Eunsol Choi
Large language models (LLMs) must often respond to highly ambiguous user requests. In such cases, the LLM's best response may be to ask a clarifying question to elicit more informa…