5 papers
Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation
Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber +1
Machine-translated benchmarks are widely used to assess the multilingual capabilities of large language models (LLMs), yet translation errors in these benchmarks remain underexplor…
Diagnosing Translated Benchmarks: An Automated Quality Assurance Study of the EU20 Benchmark Suite
Klaudia Thellmann, Bernhard Stadler, Michael Färber
Machine-translated benchmark datasets reduce costs and offer scale, but noise, loss of structure, and uneven quality weaken confidence. What matters is not merely whether we can tr…
Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
Mehdi Ali, Michael Fromm, Klaudia Thellmann +38
We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European U…
Towards Multilingual LLM Evaluation for European Languages
Klaudia Thellmann, Bernhard Stadler, Michael Fromm +8
The rise of Large Language Models (LLMs) has revolutionized natural language processing across numerous languages and tasks. However, evaluating LLM performance in a consistent and…
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?
Alexander Arno Weber, Klaudia Thellmann, Jan Ebert +4
The adaption of multilingual pre-trained LLMs into eloquent and helpful assistants is essential to facilitate their use across different language regions. In that spirit, we are th…