14 papers
When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
Wenda Xu, Sweta Agrawal, Vilém Zouhar +2
As LLMs rapidly saturate existing benchmarks, automated benchmark creation using LLMs (LLM-as-a-benchmark) -- where a model generates test inputs (LLM-as-a-testset) and evaluates o…
Fenchel-Young Variational Learning
Sophia Sklaviadis, Thomas Moellenhoff, Andre Martins +1
From a variational perspective, many statistical learning criteria involve seeking a distribution that balances empirical risk and regularization. In this paper, we broaden this pe…
TranslateGemma Technical Report
Mara Finkelstein, Isaac Caswell, Tobias Domhan +18
We present TranslateGemma, a suite of open machine translation models based on the Gemma 3 foundation models. To enhance the inherent multilingual capabilities of Gemma 3 for the t…
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings
Miguel Moura Ramos, Tomás Almeida, Daniel Vareta +4
Reinforcement learning (RL) has been proven to be an effective and robust method for training neural machine translation systems, especially when paired with powerful reward models…
Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs
HyoJung Han, Sweta Agrawal, Eleftheria Briakou
Cross-lingual alignment (CLA) aims to align multilingual representations, enabling Large Language Models (LLMs) to seamlessly transfer knowledge across languages. While intuitive,…
Déjà Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation
Julia Kreutzer, Eleftheria Briakou, Sweta Agrawal +2
Generation capabilities and language coverage of multilingual large language models (mLLMs) are advancing rapidly. However, evaluation practices for generative abilities of mLLMs a…