43 citations · 70 across the 13 of their papers we have counts for
13 papers
Will Scaling Improve Social Simulation with LLMs?
Caleb Ziems, William Held, Su Doga Karaca +3
Large Language Model (LLM) social simulations are a promising research method, but they are not yet faithful enough to be adopted widely. In this work, we investigate whether the c…
Culture Cartography: Mapping the Landscape of Cultural Knowledge
Caleb Ziems, William Held, Jane Yu +3
To serve global users safely and productively, LLMs need culture-specific knowledge that might not be learned during pre-training. How do we find such knowledge that is (1) salient…
Mind the Gap! Static and Interactive Evaluations of Large Audio Models
Minzhi Li, William Barr Held, Michael J Ryan +4
As AI chatbots become ubiquitous, voice interaction presents a compelling way to enable rapid, high-bandwidth communication for both semantic and social signals. This has driven re…
EgoNormia: Benchmarking Physical Social Norm Understanding
MohammadHossein Rezaei, Yicheng Fu, Phil Cuvin +4
Human activity is moderated by norms; however, supervision for normative reasoning is sparse, particularly where norms are physically- or socially-grounded. We thus present EGONORM…
Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors
Anisha Pal, Julia Kruk, Mansi Phute +4
Text-to-image diffusion models have impactful applications in art, design, and entertainment, yet these technologies also pose significant risks by enabling the creation and dissem…
Silent Signals, Loud Impact: LLMs for Word-Sense Disambiguation of Coded Dog Whistles
Julia Kruk, Michela Marchini, Rijul Magu +3
A dog whistle is a form of coded communication that carries a secondary meaning to specific audiences and is often weaponized for racial and socioeconomic discrimination. Dog whist…
Measuring and Addressing Indexical Bias in Information Retrieval
Caleb Ziems, William Held, Jane Dwivedi-Yu +1
Information Retrieval (IR) systems are designed to deliver relevant content, but traditional systems may not optimize rankings for fairness, neutrality, or the balance of ideas. Co…
CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies
Weiyan Shi, Ryan Li, Yutong Zhang +5
To enhance language models' cultural awareness, we design a generalizable pipeline to construct cultural knowledge bases from different online communities on a massive scale. With…
Social Skill Training with Large Language Models
Diyi Yang, Caleb Ziems, William Held +3
People rely on social skills like conflict resolution to communicate effectively and to thrive in both work and personal life. However, practice environments for social skills are…
Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future
Minzhi Li, Weiyan Shi, Caleb Ziems +1
As Natural Language Processing (NLP) systems become increasingly integrated into human social life, these technologies will need to increasingly rely on social intelligence. Althou…
Impressions: Understanding Visual Semiotics and Aesthetic Impact
Julia Kruk, Caleb Ziems, Diyi Yang
Is aesthetic impact different from beauty? Is visual salience a reflection of its capacity for effective communication? We present Impressions, a novel dataset through which to inv…
CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation
Minzhi Li, Taiwei Shi, Caleb Ziems +4
Annotated data plays a critical role in Natural Language Processing (NLP) in training models and evaluating their performance. Given recent developments in Large Language Models (L…
Racism is a Virus: Anti-Asian Hate and Counterspeech in Social Media during the COVID-19 Crisis
Bing He, Caleb Ziems, Sandeep Soni +3
The spread of COVID-19 has sparked racism and hate on social media targeted towards Asian communities. However, little is known about how racial hate spreads during a pandemic and…