5 papers · 1 filter
A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas
Pranav Narayanan Venkit, Jiayi Li, Yingfan Zhou +2
As LLMs (large language models) are increasingly used to generate synthetic personas particularly in data-limited domains such as health, privacy, and HCI, it becomes necessary to…
Can Third-parties Read Our Emotions?
Jiayi Li, Yingfan Zhou, Pranav Narayanan Venkit +4
Natural Language Processing tasks that aim to infer an author's private states, e.g., emotions and opinions, from their written text, typically rely on datasets annotated by third-…
An Audit on the Perspectives and Challenges of Hallucinations in NLP
Pranav Narayanan Venkit, Tatiana Chakravorti, Vipul Gupta +5
We audit how hallucination in large language models (LLMs) is characterized in peer-reviewed literature, using a critical examination of 103 publications across NLP research. Throu…
Sociodemographic Bias in Language Models: A Survey and Forward Path
Vipul Gupta, Pranav Narayanan Venkit, Shomir Wilson +1
Sociodemographic bias in language models (LMs) has the potential for harm when deployed in real-world settings. This paper presents a comprehensive survey of the past decade of res…
CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Bias
Vipul Gupta, Pranav Narayanan Venkit, Hugo Laurençon +2
As language models (LMs) become increasingly powerful and widely used, it is important to quantify them for sociodemographic bias with potential for harm. Prior measures of bias ar…