1 citations · 1 across the 12 of their papers we have counts for
12 papers
Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languages
A. Seza Doğruöz, Xixian Liao, Verena Blaschke +3
LLM-as-a-Judge has become the dominant evaluation paradigm for many natural language generation tasks (albeit mostly in English) due to shortcomings of conventional metrics and hig…
TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
Victor Akinode, Senyu Li, Wassim Hamidouche +3
Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We in…
AfriScience-MT: Towards Decolonizing Science in Africa through Text Translation
Idris Abdulmumin, Tajuddeen Gwadabe, Shamsuddeen Hassan Muhammad +11
The dominance of colonial languages in African education and scientific communication limits how hundreds of millions of speakers of African languages access and produce scientific…
AfrIFact: Cultural Information Retrieval, Evidence Extraction and Fact Checking for African Languages
Israel Abebe Azime, Jesujoba Oluwadara Alabi, Crystina Zhang +16
Assessing the veracity of a claim made online is a complex and important task with real-world implications. When these claims are directed at communities with limited access to inf…
Testing the Assumptions of Active Learning for Translation Tasks with Few Samples
Lorenzo Jaime Yu Flores, Cesare Spinoso di-Piano, Ori Ernst +2
Active learning (AL) is a training paradigm for selecting unlabeled samples for annotation to improve model performance on a test set, which is useful when only a limited number of…
An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages
Yinhan Lu, Gaganpreet Jhajj, Chen Zhang +2
In-context learning (ICL) allows large language models (LLMs) to adapt to new tasks from a few examples, making it promising for languages underrepresented in pre-training. Recent…
Translation as a Scalable Proxy for Multilingual Evaluation
Sheriff Issaka, Erick Rosas Gonzalez, Lieqi Liu +6
The rapid proliferation of LLMs has created a critical evaluation paradox: while LLMs claim multilingual proficiency, comprehensive non-machine-translated benchmarks exist for fewe…
MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation
Tianyi Xu, Kosei Uemura, Alfred Malengo Kondoro +6
Large language models have made substantial progress in mathematical reasoning. However, benchmark development for multilingual evaluation has lagged behind English in both difficu…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
The Token Tax: Systematic Bias in Multilingual Tokenization
Jessica M. Lundin, Ada Zhang, Nihal Karim +4
Tokenization inefficiency imposes structural disadvantages on morphologically complex, low-resource languages, inflating compute resources and depressing accuracy. We evaluate 10 l…
MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoning
Kosei Uemura, David Guzmán, Quang Phuoc Nguyen +3
Large language models excel in English but still struggle with complex reasoning in many low-resource languages (LRLs). Existing encoder-plus-decoder methods such as LangBridge and…
No Text Needed: Forecasting MT Quality and Inequity from Fertility and Metadata
Jessica M. Lundin, Ada Zhang, David Adelani +1
We show that translation quality can be predicted with surprising accuracy \textit{without ever running the translation system itself}. Using only a handful of features, token fert…