4 papers
Afri-MCQA: Multimodal Cultural Question Answering for African Languages
Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva +16
Africa is home to over one-third of the world's languages, yet remains underrepresented in AI research. We introduce Afri-MCQA, the first Multilingual Cultural Question-Answering b…
VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark
Vy Tuong Dang, An Vo, Emilio Villa-Cueva +4
We introduce VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark designed to evaluate how vision-language models (VLMs) interpret and reason over visual a…
MOMENTS: A Comprehensive Multimodal Benchmark for Theory of Mind
Emilio Villa-Cueva, S M Masrur Ahmed, Rendi Chevi +7
Understanding Theory of Mind is essential for building socially intelligent multimodal agents capable of perceiving and interpreting human behavior. We introduce MoMentS (Multimoda…
CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation
Emilio Villa-Cueva, Sholpan Bolatzhanova, Diana Turmakhan +32
Translating cultural content poses challenges for machine translation systems due to the differences in conceptualizations between cultures, where language alone may fail to convey…