9 papers
From Prediction to Intervention: Personalized Meal-Level Glucose Regulation via an LLM Agent
Mingyu Huang, Weiqing Min, Ying Jin +2
Personalized glucose regulation remains a central yet unresolved challenge in precision nutrition, as postprandial glucose response varies substantially across individuals. Existin…
TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs
Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee +2
Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In…
UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding
Ahmer Tabassum, Sarfraz Ahmad, Hasan Iqbal +3
Meaningful multilingual evaluation must test models in the target language and educational context. Urdu, spoken by more than 230 million people, lacks a broad MMLU-style benchmark…
Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
Muhammad Dehan Al Kautsar, Saeed Almheiri, Momina Ahsan +13
There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. Most Arabic benchmarks focus…
SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
Rania Elbadry, Sarfraz Ahmad, Ahmed Heakl +12
English financial NLP has advanced rapidly through benchmarks targeting earnings analysis, market sentiment, tabular reasoning, and financial question answering, yet Arabic financi…
A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding
Dilara TorunoÄlu-Selamet, Dogukan Arslan, Rodrigo Wilkens +75
Potentially idiomatic expressions (PIEs) construe meanings inherently tied to the everyday experience of a given language community. As such, they constitute an interesting challen…