9 citations · 11 across the 8 of their papers we have counts for
8 papers
Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4
Zhuozhuo Joy Liu, Farhan Samir, Mehar Bhatia +2
LLMs have been demonstrated to align with the values of Western or North American cultures. Prior work predominantly showed this effect through leveraging surveys that directly ask…
CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeaming
Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin +8
Robust, diverse, and challenging cultural knowledge benchmarks are essential for measuring our progress towards making LMs that are helpful across diverse cultures. We introduce Cu…
From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
Mehar Bhatia, Sahithya Ravi, Aditya Chinchure +2
Despite recent advancements in vision-language models, their performance remains suboptimal on images from non-western cultures due to underrepresentation in training datasets. Var…
CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledge
Yu Ying Chiu, Liwei Jiang, Maria Antoniak +7
Frontier large language models (LLMs) are developed by researchers and practitioners with skewed cultural backgrounds and on datasets with skewed sources. However, LLMs' (lack of)…
GD-COMET: A Geo-Diverse Commonsense Inference Model
Mehar Bhatia, Vered Shwartz
With the increasing integration of AI into everyday life, it's becoming crucial to design AI systems that serve users from diverse backgrounds by making them culturally aware. In t…
Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks
Anuj Saraswat, Mehar Bhatia, Yaman Kumar Singla +2
Recent studies in speech perception have been closely linked to fields of cognitive psychology, phonology, and phonetics in linguistics. During perceptual attunement, a critical an…
One to rule them all: Towards Joint Indic Language Hate Speech Detection
Mehar Bhatia, Tenzin Singhay Bhotia, Akshat Agarwal +5
This paper is a contribution to the Hate Speech and Offensive Content Identification in Indo-European Languages (HASOC) 2021 shared task. Social media today is a hotbed of toxic an…
Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems
Anubha Kabra, Mehar Bhatia, Yaman Kumar +2
Automatic scoring engines have been used for scoring approximately fifteen million test-takers in just the last three years. This number is increasing further due to COVID-19 and t…