4 papers
Can Reasoning Help Large Language Models Capture Human Annotator Disagreement?
Jingwei Ni, Yu Fan, Vilém Zouhar +6
Variation in human annotation (i.e., disagreements) is common in NLP, often reflecting important information like task subjectivity and sample ambiguity. Modeling this variation is…
Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarization
Eunjung Cho, Alexander Hoyle, Yoan Hermstrüwer
Large Language Models (LLMs) are increasingly used to generate user-tailored summaries, adapting outputs to specific stakeholders. In legal contexts, this raises important question…
How Persuasive is Your Context?
Tu Nguyen, Kevin Du, Alexander Miserlis Hoyle +1
Two central capabilities of language models (LMs) are: (i) drawing on prior knowledge about entities, which allows them to answer queries such as "What's the official language of A…
Large Language Models Struggle to Describe the Haystack without Human Help: Human-in-the-loop Evaluation of Topic Models
Zongxia Li, Lorena Calvo-Bartolomé, Alexander Hoyle +4
A common use of NLP is to facilitate the understanding of large document collections, with a shift from using traditional topic models to Large Language Models. Yet the effectivene…