2 citations · 3 across the 15 of their papers we have counts for
15 papers
TranslateGemma Technical Report
Mara Finkelstein, Isaac Caswell, Tobias Domhan +18
We present TranslateGemma, a suite of open machine translation models based on the Gemma 3 foundation models. To enhance the inherent multilingual capabilities of Gemma 3 for the t…
Pretraining Strategies using Monolingual and Parallel Data for Low-Resource Machine Translation
Idriss Nguepi Nguefack, Mara Finkelstein, Toadoum Sari Sakayo
This research article examines the effectiveness of various pretraining strategies for developing machine translation models tailored to low-resource languages. Although this work…
MetricX-25 and GemSpanEval: Google Translate Submissions to the WMT25 Evaluation Shared Task
Juraj Juraska, Tobias Domhan, Mara Finkelstein +5
In this paper, we present our submissions to the unified WMT25 Translation Evaluation Shared Task. For the Quality Score Prediction subtask, we create a new generation of MetricX w…
MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation
Parker Riley, Daniel Deutsch, Mara Finkelstein +3
Human evaluation of machine translation is in an arms race with translation model quality: as our models get better, our evaluation methods need to be improved to ensure that quali…
Generating Difficult-to-Translate Texts
Vilém Zouhar, Wenda Xu, Parker Riley +4
Machine translation benchmarks sourced from the real world are quickly obsoleted, due to most examples being easy for state-of-the-art translation models. This limits the benchmark…
Searching the Internet for Challenging Benchmarks at Scale
Wenda Xu, Vilém Zouhar, Parker Riley +3
Many static benchmarks are beginning to saturate: as models rapidly improve, they achieve near-perfect scores on fixed test sets, leaving little headroom to expose genuine model we…
WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects
Daniel Deutsch, Eleftheria Briakou, Isaac Caswell +14
As large language models (LLM) become more and more capable in languages other than English, it is important to collect benchmark datasets in order to evaluate their multilingual p…
From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set
Mara Finkelstein, Dan Deutsch, Parker Riley +3
As LLMs continue to become more powerful and versatile, human evaluation has quickly become intractable at scale and reliance on automatic metrics has become the norm. Recently, it…
Learning from others' mistakes: Finetuning machine translation models with span-level error annotations
Lily H. Zhang, Hamid Dadkhahi, Mara Finkelstein +3
Despite growing interest in incorporating feedback to improve language models, most efforts focus only on sequence-level annotations. In this work, we explore the potential of util…
MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task
Juraj Juraska, Daniel Deutsch, Mara Finkelstein +1
In this paper, we present the MetricX-24 submissions to the WMT24 Metrics Shared Task and provide details on the improvements we made over the previous version of MetricX. Our prim…
Introducing the NewsPaLM MBR and QE Dataset: LLM-Generated High-Quality Parallel Data Outperforms Traditional Web-Crawled Data
Mara Finkelstein, David Vilar, Markus Freitag
Recent research in neural machine translation (NMT) has shown that training on high-quality machine-generated data can outperform training on human-generated data. This work accomp…
Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithms
Firas Trabelsi, David Vilar, Mara Finkelstein +1
Minimum Bayes Risk (MBR) decoding is a powerful decoding strategy widely used for text generation tasks, but its quadratic computational complexity limits its practical application…
There's no Data Like Better Data: Using QE Metrics for MT Data Filtering
Jan-Thorsten Peter, David Vilar, Daniel Deutsch +3
Quality Estimation (QE), the evaluation of machine translation output without the need of explicit references, has seen big improvements in the last years with the use of neural me…
LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
Wenda Xu, Daniel Deutsch, Mara Finkelstein +6
Recent large language models (LLM) are leveraging human feedback to improve their generation quality. However, human feedback is costly to obtain, especially during inference. In t…
Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model
Christian Tomani, David Vilar, Markus Freitag +5
Maximum-a-posteriori (MAP) decoding is the most widely used decoding strategy for neural machine translation (NMT) models. The underlying assumption is that model probability corre…