3 papers
cs.CL2026
Gold vs.~Translation: Are We Measuring Language, Knowledge, or Locality in Translated Benchmarks?
Klaudia-Doris Thellmann, Bernhard Stadler, Dan Saattrup Nielsen +2
Translated versions of English-origin benchmarks such as MMLU are widely used to evaluate large language models in non-English languages, yet it remains unclear whether they measur…
cs.CL2026
Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation
Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber +1
Machine-translated benchmarks are widely used to assess the multilingual capabilities of large language models (LLMs), yet translation errors in these benchmarks remain underexplor…
cs.CL2024
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?
Alexander Arno Weber, Klaudia Thellmann, Jan Ebert +4
The adaption of multilingual pre-trained LLMs into eloquent and helpful assistants is essential to facilitate their use across different language regions. In that spirit, we are th…