Publications (35)
Adaptive Data Collection for Latin-American Community-sourced Evaluation of Stereotypes (LACES)
Guido Ivetta, Pietro Palombini, SofÃa Martinelli +5
The evaluation of societal biases in NLP models is critically hindered by a geo-cultural gap, This leaves regions such as Latin America severely underserved, making it impossible t…
MiTTenS: A Dataset for Evaluating Gender Mistranslation
Kevin Robinson, Sneha Kudugunta, Romina Stella +2
Translation systems, including foundation models capable of translation, can produce errors that result in gender mistranslation, and such errors can be especially harmful. To meas…
SoUnD Framework: Analyzing (So)cial Representation in (Un)structured (D)ata
Mark DÃaz, Sunipa Dev, Emily Reif +2
The unstructured nature of data used in foundation model development is a challenge to systematic analyses for making data use and documentation decisions. From a Responsible AI pe…
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings
Sunipa Dev, Tao Li, Jeff M Phillips +1
Language representations are known to carry stereotypical biases and, as a result, lead to biased predictions in downstream tasks. While existing methods are effective at mitigatin…
On Measuring and Mitigating Biased Inferences of Word Embeddings
Sunipa Dev, Tao Li, Jeff Phillips +1
Word embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them. We use this observa…
SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes
Mukul Bhutani, Kevin Robinson, Vinodkumar Prabhakaran +2
While generative multilingual models are rapidly being deployed, their safety and fairness evaluations are largely limited to resources collected in English. This is especially pro…
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models
Akshita Jha, Aida Davani, Chandan K. Reddy +3
Stereotype benchmark datasets are crucial to detect and mitigate social stereotypes about groups of people in NLP models. However, existing datasets are limited in size and coverag…
Re-contextualizing Fairness in NLP: The Case of India
Shaily Bhatt, Sunipa Dev, Partha Talukdar +2
Recent research has revealed undesirable biases in NLP data and models. However, these efforts focus on social disparities in West, and are not directly portable to other geo-cultu…
Cultural Re-contextualization of Fairness Research in Language Technologies in India
Shaily Bhatt, Sunipa Dev, Partha Talukdar +2
Recent research has revealed undesirable biases in NLP data and models. However, these efforts largely focus on social disparities in the West, and are not directly portable to oth…
On Measures of Biases and Harms in NLP
Sunipa Dev, Emily Sheng, Jieyu Zhao +8
Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nati…
ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation
Akshita Jha, Vinodkumar Prabhakaran, Remi Denton +5
Recent studies have shown that Text-to-Image (T2I) model generations can reflect social stereotypes present in the real world. However, existing approaches for evaluating stereotyp…
Building Socio-culturally Inclusive Stereotype Resources with Community Engagement
Sunipa Dev, Jaya Goyal, Dinesh Tewari +2
With rapid development and deployment of generative language models in global settings, there is an urgent need to also scale our measurements of harm, not just in the number and t…
MISGENDERED: Limits of Large Language Models in Understanding Pronouns
Tamanna Hossain, Sunipa Dev, Sameer Singh
Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering. Gender bias in language technologies has been widely s…
Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN
Anaelia Ovalle, Sunipa Dev, Jieyu Zhao +2
Auditing machine learning-based (ML) healthcare tools for bias is critical to preventing patient harm, especially in communities that disproportionately face health inequities. Gen…
Closed Form Word Embedding Alignment
Sunipa Dev, Safia Hassan, Jeff M. Phillips
We develop a family of techniques to align word embeddings which are derived from different source datasets or created using different mechanisms (e.g., GloVe or word2vec). Our met…
MisgenderMender: A Community-Informed Approach to Interventions for Misgendering
Tamanna Hossain, Sunipa Dev, Sameer Singh
Content Warning: This paper contains examples of misgendering and erasure that could be offensive and potentially triggering. Misgendering, the act of incorrectly addressing someon…
PaLM 2 Technical Report
Rohan Anil, Andrew M. Dai, Orhan Firat +125
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 i…
SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context
Aishwarya Verma, Laud Ammah, Olivia Nercy Ndlovu Lucas +3
Stereotype repositories are critical to assess generative AI model safety, but currently lack adequate global coverage. It is imperative to prioritize targeted expansion, strategic…
A Comprehensive Framework to Operationalize Social Stereotypes for Responsible AI Evaluations
Aida Davani, Sunipa Dev, Héctor Pérez-Urbina +1
Societal stereotypes are at the center of a myriad of responsible AI interventions targeted at reducing the generation and propagation of potentially harmful outcomes. While these…
Socially Aware Bias Measurements for Hindi Language Representations
Vijit Malik, Sunipa Dev, Akihiro Nishi +2
Language representations are efficient tools used across NLP applications, but they are strife with encoded societal biases. These biases are studied extensively, but with a primar…
JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
Jiho Jin, Junho Myung, Juhyun Oh +5
As large language models (LLMs) are increasingly deployed to users around the world, they are integrated into everyday tasks across diverse cultural contexts, from drafting persona…
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle +3
Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as…
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin +64
Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of…
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks
Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi +2
How reliably can we trust the scores obtained from social bias benchmarks as faithful indicators of problematic social biases in a given language model? In this work, we study this…
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation
Jingnong Qu, Liunian Harold Li, Jieyu Zhao +2
Disinformation has become a serious problem on social media. In particular, given their short format, visual attraction, and humorous nature, memes have a significant advantage in…
A Unified Framework to Quantify Cultural Intelligence of AI
Sunipa Dev, Vinodkumar Prabhakaran, Rutledge Chin Feman +16
As generative AI technologies are increasingly being launched across the globe, assessing their competence to operate in different cultural contexts is exigently becoming a priorit…
Representation Learning for Resource-Constrained Keyphrase Generation
Di Wu, Wasi Uddin Ahmad, Sunipa Dev +1
State-of-the-art keyphrase generation methods generally depend on large annotated datasets, limiting their performance in domains with limited annotated data. To overcome this chal…
Towards Geo-Culturally Grounded LLM Generations
Piyawat Lertvittayakumjorn, David Kinney, Vinodkumar Prabhakaran +2
Generative large language models (LLMs) have demonstrated gaps in diverse cultural awareness across the globe. We investigate the effect of retrieval augmented generation and searc…
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
Sunipa Dev
High-dimensional representations for words, text, images, knowledge graphs and other structured data are commonly used in different paradigms of machine learning and data mining. T…
Attenuating Bias in Word Vectors
Sunipa Dev, Jeff Phillips
Word vector representations are well developed tools for various NLP and Machine Learning tasks and are known to retain significant semantic and syntactic structure of languages. B…
Scaling Cultural Resources for Improving Generative Models
Hayk Stepanyan, Aishwarya Verma, Andrew Zaldivar +5
Generative models are known to have reduced performance in different global cultural contexts and languages. While continual data updates have been commonly conducted to improve ov…
GeniL: A Multilingual Dataset on Generalizing Language
Aida Mostafazadeh Davani, Sagar Gubbi, Sunipa Dev +2
Generative language models are transforming our digital ecosystem, but they often inherit societal biases, for instance stereotypes associating certain attributes with specific ide…
Cultural Authenticity: Comparing LLM Cultural Representations to Native Human Expectations
Erin MacMurray van Liemt, Aida Davani, Sinchana Kumbale +2
Cultural representation in Large Language Model (LLM) outputs has primarily been evaluated through the proxies of cultural diversity and factual accuracy. However, a crucial gap re…
VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations
Archit Rathore, Sunipa Dev, Jeff M. Phillips +6
Word vector embeddings have been shown to contain and amplify biases in data they are extracted from. Consequently, many techniques have been proposed to identify, mitigate, and at…
Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations
Myra Cheng, Vinodkumar Prabhakaran, Alice Oh +5
Generative AI models ought to be useful and safe across cross-cultural contexts. One critical step toward this goal is understanding how AI models adhere to sociocultural norms. Wh…