131 citations · 319 across the 29 of their papers we have counts for
32 papers
Turning Speech Language Models into Multilingual Listeners
Tolúlopé Ògúnrèmí, Dan Jurafsky, Chris Manning +2
Speech Language Models (SLMs) that understand spoken language questions support only a few high-resource languages, limiting access to millions of people worldwide. This gap stems…
LLMs as Oracles: Reliance on LLMs for Subjective Personal Questions
Myra Cheng, Lujain Ibrahim, Grace Liu +5
We characterize how people are turning to LLMs as oracles: all-knowing authorities on subjective personal questions. Motivated by risks to users' autonomy and well-being, we develo…
string2string Studio: An Interactive, In-Browser Platform for String-to-String Algorithms
Mirac Suzgun, James Zou, Stuart M. Shieber +1
We present string2string Studio, an interactive in-browser platform for string-to-string analysis across natural language processing, computational biology, and the digital humanit…
You Talkin to Me?: A Network Analysis of Gendered Speaker-Addressee Patterns in Film Screenplays
Samin Khan, Camilla Griffiths, Shrikanth Narayanan +2
Objective: This paper investigates the gendered structure of speaker addressee relationships in film dialogue, asking not merely who speaks, but who is spoken to and how conversati…
Algorithmic Monocultures in Hiring
Rishi Bommasani, Sarah H. Bana, Kathleen A. Creel +2
Many employers screen job applicants with algorithms built by the same few algorithm vendors. We hypothesize that algorithmic monoculture leads to the same individuals and members…
Verbalizing LLMs' assumptions to explain and control sycophancy
Myra Cheng, Isabel Sieh, Humishka Zope +7
LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment. We hypothesize that this behavior aris…
Beyond Tokens: Concept-Level Training Objectives for LLMs
Laya Iyer, Pranav Somani, Alice Guo +2
The next-token prediction (NTP) objective has been foundational in the development of modern large language models (LLMs), driving advances in fluency and generalization. However,…
Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs
Myra Cheng, Robert D. Hawkins, Dan Jurafsky
Large language models (LLMs) frequently fail to challenge users' harmful beliefs in domains ranging from medical advice to social reasoning. We argue that these failures can be und…
Generation Space Size: Understanding and Calibrating Open-Endedness of LLM Generations
Sunny Yu, Ahmad Jabbar, Robert Hawkins +2
Different open-ended generation tasks require different degrees of output diversity. However, current LLMs are often miscalibrated. They collapse to overly homogeneous outputs for…
Attention to Non-Adopters
Kaitlyn Zhou, Kristina Gligorić, Myra Cheng +7
Although language model-based chat systems are increasingly used in daily life, most Americans remain non-adopters of chat-based LLMs -- as of June 2025, 66% had never used ChatGPT…
Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence
Myra Cheng, Cinoo Lee, Pranav Khadpe +3
Both the general public and academic communities have raised concerns about sycophancy, the phenomenon of artificial intelligence (AI) excessively agreeing with or flattering users…
The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varieties
William Chen, Chutong Meng, Jiatong Shi +10
Recent improvements in multilingual ASR have not been equally distributed across languages and language varieties. To advance state-of-the-art (SOTA) ASR models, we present the Int…
Artificial Intelligence for Food Innovation
Bianca Datta, Markus J. Buehler, Yvonne Chow +14
Global food systems must deliver nutritious, sustainable foods while sharply reducing environmental impact. Yet, food innovation remains slow, empirical, and fragmented. Artificial…
Humans overrely on overconfident language models, across languages
Neil Rathi, Dan Jurafsky, Kaitlyn Zhou
As large language models (LLMs) are deployed globally, it is crucial that their responses are calibrated across languages to accurately convey uncertainty and limitations. Prior wo…
ELEPHANT: Measuring and understanding social sycophancy in LLMs
Myra Cheng, Sunny Yu, Cinoo Lee +3
LLMs are known to exhibit sycophancy: agreeing with and flattering users, even at the cost of correctness. Prior work measures sycophancy only as direct agreement with users' expli…
Cooking Up Creativity: Enhancing LLM Creativity through Structured Recombination
Moran Mizrahi, Chen Shani, Gabriel Stanovsky +2
Large Language Models (LLMs) excel at many tasks, yet they struggle to produce truly creative, diverse ideas. In this paper, we introduce a novel approach that enhances LLM creativ…
What can large language models do for sustainable food?
Anna T. Thomas, Adam Yee, Andrew Mayne +3
Food systems are responsible for a third of human-caused greenhouse gas emissions. We investigate what Large Language Models (LLMs) can contribute to reducing the environmental imp…
CTC-DRO: Robust Optimization for Reducing Language Disparities in Speech Recognition
Martijn Bartelds, Ananjan Nandi, Moussa Koulako Bala Doumbouya +3
Modern deep learning models often achieve high overall performance, but consistently fail on specific subgroups. Group distributionally robust optimization (group DRO) addresses th…
Belief in the Machine: Investigating Epistemological Blind Spots of Language Models
Mirac Suzgun, Tayfun Gur, Federico Bianchi +4
As language models (LMs) become integral to fields like healthcare, law, and journalism, their ability to differentiate between fact, belief, and knowledge is essential for reliabl…
Data Checklist: On Unit-Testing Datasets with Usable Information
Heidi C. Zhang, Shabnam Behzad, Kawin Ethayarajh +1
Model checklists (Ribeiro et al., 2020) have emerged as a useful tool for understanding the behavior of LLMs, analogous to unit-testing in software engineering. However, despite da…
Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren +3
The ability to communicate uncertainty, risk, and limitation is crucial for the safety of large language models. However, current evaluations of these abilities rely on simple cali…
ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets
Jiatong Shi, Shih-Heng Wang, William Chen +8
ML-SUPERB evaluates self-supervised learning (SSL) models on the tasks of language identification and automatic speech recognition (ASR). This benchmark treats the models as featur…
NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps
Kristina Gligoric, Myra Cheng, Lucia Zheng +2
The use of words to convey speaker's intent is traditionally distinguished from the `mention' of words for quoting what someone said, or pointing out properties of a word. Here we…
Dialect prejudice predicts AI decisions about people's character, employability, and criminality
Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky +1
Hundreds of millions of people now interact with language models, with uses ranging from serving as a writing aid to informing hiring decisions. Yet these language models are known…