7 papers · 1 filter
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Instruction-Guided Poetry Generation in Arabic and Its Dialects
Abdelrahman Sadallah, Kareem Elozeiri, Mervat Abassy +5
Poetry has long been a central art form for Arabic speakers, serving as a powerful medium of expression and cultural identity. While modern Arabic speakers continue to value poetry…
The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authors
Abdelrahman Sadallah, Tim Baumgärtner, Iryna Gurevych +1
Providing constructive feedback to paper authors is a core component of peer review. With reviewers increasingly having less time to perform reviews, automated support systems are…
Commonsense Reasoning in Arab Culture
Abdelrahman Sadallah, Junior Cedric Tonga, Khalid Almubarak +7
Despite progress in Arabic large language models, such as Jais and AceGPT, their evaluation on commonsense reasoning has largely relied on machine-translated datasets, which lack c…
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev +12
The rapid proliferation of large language models (LLMs) has stimulated researchers to seek effective and efficient approaches to deal with LLM hallucinations and low-quality output…
What Makes Cryptic Crosswords Challenging for LLMs?
Abdelrahman Sadallah, Daria Kotova, Ekaterina Kochmar
Cryptic crosswords are puzzles that rely on general knowledge and the solver's ability to manipulate language on different levels, dealing with various types of wordplay. Previous…