3 papers
cs.CL2026
Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Results
Jan-Thorsten Peter, David Vilar, Tobias Domhan +2
Most current large language models (LLMs) support a wide variety of languages in addition to English, including high-resource languages (e.g. German, Chinese, French), as well as l…
cs.CL2025
You Cannot Feed Two Birds with One Score: the Accuracy-Naturalness Tradeoff in Translation
Gergely Flamich, David Vilar, Jan-Thorsten Peter +1
The goal of translation, be it by human or by machine, is, given some text in a source language, to produce text in a target language that simultaneously 1) preserves the meaning o…
cs.CL2025
Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation
Muhammed Yusuf Kocyigit, Eleftheria Briakou, Daniel Deutsch +3
Data contamination -- the accidental consumption of evaluation examples within the pre-training data -- can undermine the validity of evaluation benchmarks. In this paper, we prese…