3 citations · 8 across the 12 of their papers we have counts for
15 papers
Cultural Authenticity: Comparing LLM Cultural Representations to Native Human Expectations
Erin MacMurray van Liemt, Aida Davani, Sinchana Kumbale +2
Cultural representation in Large Language Model (LLM) outputs has primarily been evaluated through the proxies of cultural diversity and factual accuracy. However, a crucial gap re…
A Unified Framework to Quantify Cultural Intelligence of AI
Sunipa Dev, Vinodkumar Prabhakaran, Rutledge Chin Feman +16
As generative AI technologies are increasingly being launched across the globe, assessing their competence to operate in different cultural contexts is exigently becoming a priorit…
Humanlike AI Design Increases Anthropomorphism but Yields Divergent Outcomes on Engagement and Trust Globally
Robin Schimmelpfennig, Mark Díaz, Vinodkumar Prabhakaran +1
Over a billion users globally interact with AI systems engineered to mimic human traits. This development raises concerns that anthropomorphism, the attribution of human characteri…
"Just a strange pic": Evaluating 'safety' in GenAI Image safety annotation tasks from diverse annotators' perspectives
Ding Wang, Mark Díaz, Charvi Rastogi +10
Understanding what constitutes safety in AI-generated content is complex. While developers often rely on predefined taxonomies, real-world safety judgments also involve personal, s…
Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models
Charvi Rastogi, Tian Huey Teh, Pushkar Mishra +10
Current text-to-image (T2I) models often fail to account for diverse human experiences, leading to misaligned systems. We advocate for pluralistic alignment, where an AI understand…
Risks of Cultural Erasure in Large Language Models
Rida Qadri, Aida M. Davani, Kevin Robinson +1
Large language models are increasingly being integrated into applications that shape the production and discovery of societal knowledge such as search, online education, and travel…
A Comprehensive Framework to Operationalize Social Stereotypes for Responsible AI Evaluations
Aida Davani, Sunipa Dev, Héctor Pérez-Urbina +1
Societal stereotypes are at the center of a myriad of responsible AI interventions targeted at reducing the generation and propagation of potentially harmful outcomes. While these…
Insights on Disagreement Patterns in Multimodal Safety Perception across Diverse Rater Groups
Charvi Rastogi, Tian Huey Teh, Pushkar Mishra +10
AI systems crucially rely on human ratings, but these ratings are often aggregated, obscuring the inherent diversity of perspectives in real-world phenomenon. This is particularly…
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation
Aida Mostafazadeh Davani, Mark Díaz, Dylan Baker +1
While human annotations play a crucial role in language technologies, annotator subjectivity has long been overlooked in data collection. Recent studies that have critically examin…
GeniL: A Multilingual Dataset on Generalizing Language
Aida Mostafazadeh Davani, Sagar Gubbi, Sunipa Dev +2
Generative language models are transforming our digital ecosystem, but they often inherit societal biases, for instance stereotypes associating certain attributes with specific ide…
Disentangling Perceptions of Offensiveness: Cultural and Moral Correlates
Aida Davani, Mark Díaz, Dylan Baker +1
Perception of offensiveness is inherently subjective, shaped by the lived experiences and socio-cultural values of the perceivers. Recent years have seen substantial efforts to bui…
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models
Akshita Jha, Aida Davani, Chandan K. Reddy +3
Stereotype benchmark datasets are crucial to detect and mitigate social stereotypes about groups of people in NLP models. However, existing datasets are limited in size and coverag…
The Moral Foundations Reddit Corpus
Jackson Trager, Alireza S. Ziabari, Elnaz Rahmati +15
Moral framing and sentiment can affect a variety of online and offline behaviors, including donation, environmental action, political engagement, and protest. Various computational…
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
Aida Mostafazadeh Davani, Mark Díaz, Vinodkumar Prabhakaran
Majority voting and averaging are common approaches employed to resolve annotator disagreements and derive single ground truth labels from multiple annotations. However, annotators…
On Releasing Annotator-Level Labels and Information in Datasets
Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, Mark Díaz
A common practice in building NLP datasets, especially using crowd-sourced annotations, involves obtaining multiple annotator judgements on the same data instances, which are then…