8 papers
TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs
Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee +2
Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In…
UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding
Ahmer Tabassum, Sarfraz Ahmad, Hasan Iqbal +3
Meaningful multilingual evaluation must test models in the target language and educational context. Urdu, spoken by more than 230 million people, lacks a broad MMLU-style benchmark…
Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
Muhammad Dehan Al Kautsar, Saeed Almheiri, Momina Ahsan +13
There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. Most Arabic benchmarks focus…
SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
Rania Elbadry, Sarfraz Ahmad, Ahmed Heakl +12
English financial NLP has advanced rapidly through benchmarks targeting earnings analysis, market sentiment, tabular reasoning, and financial question answering, yet Arabic financi…
A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding
Dilara TorunoÄlu-Selamet, Dogukan Arslan, Rodrigo Wilkens +75
Potentially idiomatic expressions (PIEs) construe meanings inherently tied to the everyday experience of a given language community. As such, they constitute an interesting challen…
iBitter-Stack: A Multi-Representation Ensemble Learning Model for Accurate Bitter Peptide Identification
Sarfraz Ahmad, Momina Ahsan, Muhammad Nabeel Asim +2
The identification of bitter peptides is crucial in various domains, including food science, drug discovery, and biochemical research. These peptides not only contribute to the und…