25 citations · 77 across the 74 of their papers we have counts for
78 papers
Halluscoring 2026: The first shared task on llms hallucination detection and answer verification
Aisha Alansari, Abdessalam Bouchekif, Ahmed Hasanaath +9
We present HalluScoring 2026, a shared task for evaluating hallucination detection and factual verification in Arabic question answering under challenging generalization settings.…
CARDAMOM: A Micro-Dialectal Arabic Speech Dataset for ASR
Bashar Talafha, Samar M. Magdy, Aisha Alansari +36
We present Cardamom, a micro-dialectal Arabic speech dataset designed to support fine-grained evaluation and adaptation of automatic speech recognition (ASR) systems. Community-cur…
NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task
Peter Sullivan, Bashar Talafha, Ahmed Ashraf +11
NADI 2026 is the seventh edition of the Nuanced Arabic Dialect Identification (NADI) shared task series and the second dedicated to multidialectal Arabic speech processing. This ed…
AlexandriaX 2026: The First Shared Task on Dialectal Arabic Machine Translation
Abdellah El Mekki, AbdelRahim A. Elmadany, Samar M. Magdy +6
Dialectal Arabic machine translation (MT) remains challenging despite recent progress in Arabic language technologies, particularly because effective translation requires modeling…
Bulbul: A Dataset for Dialectal Arabic Speech Recognition
Ahmed Ashraf, Aisha Alansari, Fadel Al Abbas +30
Arabic automatic speech recognition (ASR) faces unique challenges due to diglossia, extensive regional dialect variation, and limited speech resources. Existing speech datasets oft…
MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning
Abdelrahman Abdallah, AbdelRahim A. Elmadany, Sameh Al Natour +3
Financial and tabular question answering requires more than fluent reasoning: answers must be grounded in the exact facts, formulas, units, signs, and scales that support them. A s…
LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation
Samar M. Magdy, Fakhraddin Alwajih, Abdellah El Mekki +2
Existing MT evaluation frameworks, including automatic metrics and human evaluation schemes such as Multidimensional Quality Metrics (MQM), are largely language-agnostic. However,…
Arab Voices: Mapping Standard and Dialectal Arabic Speech Technology
Peter Sullivan, AbdelRahim Elmadany, Alcides Alcoba Inciarte +1
Dialectal Arabic (DA) speech data vary widely in domain coverage, dialect labeling practices, and recording conditions, complicating cross-dataset comparison and model evaluation.…
TEMPO: A Realistic Multi-Domain Benchmark for Temporal Reasoning-Intensive Retrieval
Abdelrahman Abdallah, Mohammed Ali, Muhammad Abdul-Mageed +1
Existing temporal QA benchmarks focus on simple fact-seeking queries from news corpora, while reasoning-intensive retrieval benchmarks lack temporal grounding. However, real-world…
AfroScope: A Framework for Studying the Linguistic Landscape of Africa
Sang Yun Kwon, AbdelRahim Elmadany, Muhammad Abdul-Mageed
Language Identification (LID), the task of determining the language of a given text, is a fundamental preprocessing step that shapes the reliability of downstream NLP applications.…
Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs
Abdellah El Mekki, Samar M. Magdy, Houdaifa Atou +44
Arabic is a highly diglossic language where most daily communication occurs in regional dialects rather than Modern Standard Arabic (MSA). Despite this, machine translation (MT) sy…
Zero-Shot Context-Aware ASR for Diverse Arabic Varieties
Bashar Talafha, Amin Abu Alhassan, Muhammad Abdul-Mageed
Zero-shot ASR for Arabic remains challenging: while multilingual models perform well on Modern Standard Arabic (MSA), error rates rise sharply on dialectal and accented speech due…
Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMs
Muhammed Saeed, Muhammad Abdul-mageed, Shady Shehata
Large language models (LLMs) are widely deployed for open-ended communication, yet most bias evaluations still rely on English, classification-style tasks. We introduce \corpusname…
EduAdapt: A Question Answer Benchmark Dataset for Evaluating Grade-Level Adaptability in LLMs
Numaan Naeem, Abdellah El Mekki, Muhammad Abdul-Mageed
Large language models (LLMs) are transforming education by answering questions, explaining complex concepts, and generating content across a wide range of subjects. Despite strong…
AraHealthQA 2025: The First Shared Task on Arabic Health Question Answering
Hassan Alhuzali, Walid Al-Eisawi, Muhammad Abdul-Mageed +9
We introduce AraHealthQA 2025, the Comprehensive Arabic Health Question Answering Shared Task, held in conjunction with ArabicNLP 2025 (co-located with EMNLP 2025). This shared tas…
NADI 2025: The First Multidialectal Arabic Speech Processing Shared Task
Bashar Talafha, Hawau Olamide Toyin, Peter Sullivan +9
We present the findings of the sixth Nuanced Arabic Dialect Identification (NADI 2025) Shared Task, which focused on Arabic speech dialect processing across three subtasks: spoken…
PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culture
Fakhraddin Alwajih, Abdellah El Mekki, Hamdy Mubarak +3
Large Language Models (LLMs) inherently reflect the vast data distributions they encounter during their pre-training phase. As this data is predominantly sourced from the web, ther…
Beyond Content: How Grammatical Gender Shapes Visual Representation in Text-to-Image Models
Muhammed Saeed, Shaina Raza, Ashmal Vayani +3
Research on bias in Text-to-Image (T2I) models has primarily focused on demographic representation and stereotypical attributes, overlooking a fundamental question: how does gramma…
On Barriers to Archival Audio Processing
Peter Sullivan, Muhammad Abdul-Mageed
In this study, we leverage a unique UNESCO collection of mid-20th century radio recordings to probe the robustness of modern off-the-shelf language identification (LID) and speaker…
DefenderBench: A Toolkit for Evaluating Language Agents in Cybersecurity Environments
Chiyu Zhang, Marc-Alexandre Cote, Michael Albada +6
Large language model (LLM) agents have shown impressive capabilities in human language comprehension and reasoning, yet their potential in cybersecurity remains underexplored. We i…
Pearl: A Multimodal Culturally-Aware Arabic Instruction Dataset
Fakhraddin Alwajih, Samar M. Magdy, Abdellah El Mekki +34
Mainstream large vision-language models (LVLMs) inherently encode cultural biases, highlighting the need for diverse multimodal datasets. To address this gap, we introduce PEARL, a…
Voice of a Continent: Mapping Africa's Speech Technology Frontier
AbdelRahim Elmadany, Sang Yun Kwon, Hawau Olamide Toyin +3
Africa's rich linguistic diversity remains significantly underrepresented in speech technologies, creating barriers to digital inclusion. To alleviate this challenge, we systematic…
NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communities
Abdellah El Mekki, Houdaifa Atou, Omer Nacar +2
Enhancing the linguistic capabilities of Large Language Models (LLMs) to include low-resource languages is a critical research area. Current research directions predominantly rely…
Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs
Fakhraddin Alwajih, Abdellah El Mekki, Samar Mohamed Magdy +41
As large language models (LLMs) become increasingly integrated into daily life, ensuring their cultural sensitivity and inclusivity is paramount. We introduce our dataset, a year-l…