6 papers
Characterizing Cultural Localization in AI-Generated Stories
Shaily Bhatt, Supriti Vijay, Jeremiah Milbauer +1
The global use of artificial intelligence has increased interest in assessing the ability to generate culturally localized content, including stories. Cultural localization in stor…
TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories
Kirti Bhagat, Shaily Bhatt, Athul Velagapudi +3
Millions of users across the globe turn to AI chatbots for their creative needs, inviting widespread interest in understanding how they represent diverse cultures. However, evaluat…
Who Speaks Matters: Analysing the Influence of the Speaker's Ethnicity on Hate Classification
Ananya Malik, Kartik Sharma, Shaily Bhatt +1
Large Language Models (LLMs) offer a lucrative promise for scalable content moderation, including hate speech detection. However, they are also known to be brittle and biased again…
Culture is Everywhere: A Call for Intentionally Cultural Evaluation
Juhyun Oh, Inha Cha, Michael Saxon +3
The prevailing ``trivia-centered paradigm'' for evaluating the cultural alignment of large language models (LLMs) is increasingly inadequate as these models become more advanced an…
Research Borderlands: Analysing Writing Across Research Cultures
Shaily Bhatt, Tal August, Maria Antoniak
Improving cultural competence of language technologies is important. However most recent works rarely engage with the communities they study, and instead rely on synthetic setups a…
Extrinsic Evaluation of Cultural Competence in Large Language Models
Shaily Bhatt, Fernando Diaz
Productive interactions between diverse users and language technologies require outputs from the latter to be culturally relevant and sensitive. Prior works have evaluated models'…