110 citations · 226 across the 11 of their papers we have counts for
12 papers
StanceBench: A Benchmark for Audio LLM-Based Interpersonal Stance Evaluation from Speech
Yuzhe Wang, Thomas Thebaud, Jennifer Hu +5
Speech-to-speech dialogue models increasingly depend on prosody and interactional nuance to convey social intent, yet benchmarks for these cues remain limited. We introduce StanceB…
Prompting is not a substitute for probability measurements in large language models
Jennifer Hu, Roger Levy
Prompting is now a dominant method for evaluating the linguistic knowledge of large language models (LLMs). While other methods directly read out models' probability distributions…
Expectations over Unspoken Alternatives Predict Pragmatic Inferences
Jennifer Hu, Roger Levy, Judith Degen +1
Scalar inferences (SI) are a signature example of how humans interpret language based on unspoken alternatives. While empirical studies have demonstrated that human SI rates are hi…
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
Pei Zhou, Andrew Zhu, Jennifer Hu +5
We propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment. Dungeons and Dragons (D&D), a role-playing game, pro…
A fine-grained comparison of pragmatic language understanding in humans and language models
Jennifer Hu, Sammy Floyd, Olessia Jouravlev +2
Pragmatics and non-literal language understanding are essential to human communication, and present a long-standing challenge for artificial language models. We perform a fine-grai…
Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches
Daniel Fried, Nicholas Tomlin, Jennifer Hu +2
People rely heavily on context to enrich meaning beyond what is literally said, enabling concise but effective communication. To interact successfully and naturally with people, us…
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models
Yiwen Wang, Jennifer Hu, Roger Levy +1
Prior work has shown that structural supervision helps English language models learn generalizations about syntactic phenomena such as subject-verb agreement. However, it remains u…
Scalable pragmatic communication via self-supervision
Jennifer Hu, Roger Levy, Noga Zaslavsky
Models of context-sensitive communication often use the Rational Speech Act framework (RSA; Frank & Goodman, 2012), which formulates listeners and speakers in a cooperative reasoni…
On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior
Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu +2
Human reading behavior is tuned to the statistics of natural language: the time it takes human subjects to read a word can be predicted from estimates of the word's probability in…
A Systematic Assessment of Syntactic Generalization in Neural Language Models
Jennifer Hu, Jon Gauthier, Peng Qian +2
While state-of-the-art neural network models continue to achieve lower perplexity scores on language modeling benchmarks, it remains unknown whether optimizing for broad-coverage p…
A Rate-Distortion view of human pragmatic reasoning
Noga Zaslavsky, Jennifer Hu, Roger P. Levy
What computational principles underlie human pragmatic reasoning? A prominent approach to pragmatics is the Rational Speech Act (RSA) framework, which formulates pragmatic reasonin…
Generating Bilingual Pragmatic Color References
Will Monroe, Jennifer Hu, Andrew Jong +1
Contextual influences on language often exhibit substantial cross-lingual regularities; for example, we are more verbose in situations that require finer distinctions. However, the…