4 papers
Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning
Mena Attia, Mona Diab, Thamar Solorio
Figurative language is deeply culturally embedded; fluent use requires not just linguistic competence but cultural immersion. We ask whether LLMs can learn this link: does fine-tun…
CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries
Vasudha Varadarajan, Akhila Yerukola, Mona T. Diab +1
To interact with users fairly and without stereotyping, AI models must display cultural competency, i.e., the ability to infer and adapt to a user's implicitly signaled cultural va…
Beyond Understanding: Evaluating the Pragmatic Gap in LLMs' Cultural Processing of Figurative Language
Mena Attia, Aashiq Muhamed, Mai Alkhamissi +2
We present a comprehensive evaluation of the ability of large language models (LLMs) to process culturally grounded language, specifically to understand and pragmatically use figur…
Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
Mai AlKhamissi, Yunze Xiao, Badr AlKhamissi +1
Cultural evaluation of large language models has become increasingly important, yet current benchmarks often reduce culture to static facts or homogeneous values. This view conflic…