62 citations · 104 across the 24 of their papers we have counts for
22 papers · 1 filter
Will Scaling Improve Social Simulation with LLMs?
Caleb Ziems, William Held, Su Doga Karaca +3
Large Language Model (LLM) social simulations are a promising research method, but they are not yet faithful enough to be adopted widely. In this work, we investigate whether the c…
Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment
Woody Haosheng Gan, William Held, Diyi Yang
The rapid proliferation of large audio models (LAMs) demands efficient approaches for model comparison, yet comprehensive benchmarks are costly. To fill this gap, we investigate wh…
Culture Cartography: Mapping the Landscape of Cultural Knowledge
Caleb Ziems, William Held, Jane Yu +3
To serve global users safely and productively, LLMs need culture-specific knowledge that might not be learned during pre-training. How do we find such knowledge that is (1) salient…
Relative Scaling Laws for LLMs
William Held, David Hall, Percy Liang +1
Scaling laws describe how language models improve with additional data, parameters, and compute. While widely used, they are typically measured on aggregate test sets. Aggregate ev…
AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation
Potsawee Manakul, Woody Haosheng Gan, Michael J. Ryan +5
Current speech evaluation suffers from two critical limitations: the need and difficulty of designing specialized systems targeting individual audio characteristics, and poor corre…
SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs
Michael J Ryan, Omar Shaikh, Aditri Bhagirath +3
Recent calls for pluralistic alignment of Large Language Models (LLMs) encourage adapting models to diverse user preferences. However, most prior work on personalized reward models…
Mind the Gap! Static and Interactive Evaluations of Large Audio Models
Minzhi Li, William Barr Held, Michael J Ryan +4
As AI chatbots become ubiquitous, voice interaction presents a compelling way to enable rapid, high-bandwidth communication for both semantic and social signals. This has driven re…
Optimizing Pretraining Data Mixtures with LLM-Estimated Utility
William Held, Bhargavi Paranjape, Punit Singh Koura +3
Large Language Models improve with increasing amounts of high-quality training data. However, leveraging larger datasets requires balancing quality, quantity, and diversity across…
Distilling an End-to-End Voice Assistant Without Instruction Training Data
William Held, Ella Li, Michael Ryan +3
Voice assistants, such as Siri and Google Assistant, typically model audio and text separately, resulting in lost speech information and increased complexity. Recent efforts to add…
Social Skill Training with Large Language Models
Diyi Yang, Caleb Ziems, William Held +3
People rely on social skills like conflict resolution to communicate effectively and to thrive in both work and personal life. However, practice environments for social skills are…
Unintended Impacts of LLM Alignment on Global Representation
Michael J. Ryan, William Held, Diyi Yang
Before being deployed for user-facing applications, developers align Large Language Models (LLMs) to user preferences through a variety of procedures, such as Reinforcement Learnin…
A Material Lens on Coloniality in NLP
William Held, Camille Harris, Michael Best +1
Coloniality, the continuation of colonial harms beyond "official" colonization, has pervasive effects across society and scientific fields. Natural Language Processing (NLP) is no…
Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Zedian Xiao, William Held, Yanchen Liu +1
Large Language Models (LLMs) are trained on corpora disproportionally weighted in favor of Standard American English. As a result, speakers of other dialects experience significant…
Modeling Cross-Cultural Pragmatic Inference with Codenames Duet
Omar Shaikh, Caleb Ziems, William Held +3
Pragmatic reference enables efficient interpersonal communication. Prior work uses simple reference games to test models of pragmatic reasoning, often with unidentified speakers an…
TADA: Task-Agnostic Dialect Adapters for English
Will Held, Caleb Ziems, Diyi Yang
Large Language Models, the dominant starting point for Natural Language Processing (NLP) applications, fail at a higher rate for speakers of English dialects other than Standard Am…
DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules
Yanchen Liu, William Held, Diyi Yang
Existing large language models (LLMs) that mainly focus on Standard American English (SAE) often lead to significantly worse performance when being applied to other English dialect…
Can Large Language Models Transform Computational Social Science?
Caleb Ziems, William Held, Omar Shaikh +3
Large Language Models (LLMs) are capable of successfully performing many language processing tasks zero-shot (without training data). If zero-shot LLMs can also reliably classify a…
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Omar Shaikh, Hongxin Zhang, William Held +2
Generating a Chain of Thought (CoT) has been shown to consistently improve large language model (LLM) performance on a wide range of NLP tasks. However, prior work has mainly focus…
DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue
William Held, Christopher Hidey, Fei Liu +4
Modern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands. However, prior work has demonstrated that semantic parsing is a d…
Multi-VALUE: A Framework for Cross-Dialectal English NLP
Caleb Ziems, William Held, Jingfeng Yang +3
Dialect differences caused by regional, social, and economic factors cause performance discrepancies for many groups of language technology users. Inclusive and equitable language…
Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers
William Held, Diyi Yang
Multilingual transformer-based models demonstrate remarkable zero and few-shot transfer across languages by learning and reusing language-agnostic features. However, as a fixed-siz…
Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference
William Held, Dan Iter, Dan Jurafsky
Performing event and entity coreference resolution across documents vastly increases the number of candidate mentions, making it intractable to do the full pairwise compariso…