5 papers
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan +30
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking pract…
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Min Sen Tan, Zachary Kit Chun Choy, Syed Ali Redha Alsagoff +4
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizin…
Emergence of Context Characteristics Sensitivity in Large Language Models
Nadya Yuki Wangsajaya, Haeun Yu, Isabelle Augenstein
During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query. While prior work has studied how co…
Does Less Hallucination Mean Less Creativity? An Empirical Investigation in LLMs
Mohor Banerjee, Nadya Yuki Wangsajaya, Syed Ali Redha Alsagoff +3
Large Language Models (LLMs) exhibit remarkable capabilities in natural language understanding and reasoning, but suffer from hallucination: the generation of factually incorrect c…
MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty +32
Large language models (LLMs) are now used worldwide, yet their multimodal understanding and reasoning often degrade outside Western, high-resource settings. We propose MMA-ASIA, a…