5 papers
Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests
Manar Ali, Judith Sieker, Sina Zarrieà +1
In human conversation, both interlocutors play an active role in maintaining mutual understanding. When listeners are uncertain about what speakers mean, for example, they can requ…
How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models
Judith Sieker, Sina ZarrieÃ
Large language models (LLMs) are increasingly studied as repositories of linguistic knowledge. In this line of work, models are commonly evaluated both as generators of language an…
LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High
Judith Sieker, Clara Lachenmaier, Sina ZarrieÃ
This paper examines how LLMs handle false presuppositions and whether certain linguistic factors influence their responses to falsely presupposed content. Presuppositions subtly in…
Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
Raha Askari, Sina ZarrieÃ, Ãzge Alacam +1
Implicit meanings are integral to human communication, making it essential for language models to be capable of identifying and interpreting them. Grice (1975) proposed a set of co…
Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
Clara Lachenmaier, Judith Sieker, Sina ZarrieÃ
Communication among humans relies on conversational grounding, allowing interlocutors to reach mutual understanding even when they do not have perfect knowledge and must resolve di…