4 papers · 1 filter
Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations
Myra Cheng, Vinodkumar Prabhakaran, Alice Oh +5
Generative AI models ought to be useful and safe across cross-cultural contexts. One critical step toward this goal is understanding how AI models adhere to sociocultural norms. Wh…
Adaptive Data Collection for Latin-American Community-sourced Evaluation of Stereotypes (LACES)
Guido Ivetta, Pietro Palombini, SofÃa Martinelli +5
The evaluation of societal biases in NLP models is critically hindered by a geo-cultural gap, This leaves regions such as Latin America severely underserved, making it impossible t…
Scaling Cultural Resources for Improving Generative Models
Hayk Stepanyan, Aishwarya Verma, Andrew Zaldivar +5
Generative models are known to have reduced performance in different global cultural contexts and languages. While continual data updates have been commonly conducted to improve ov…
A Comprehensive Framework to Operationalize Social Stereotypes for Responsible AI Evaluations
Aida Davani, Sunipa Dev, Héctor Pérez-Urbina +1
Societal stereotypes are at the center of a myriad of responsible AI interventions targeted at reducing the generation and propagation of potentially harmful outcomes. While these…