124 citations · 446 across the 25 of their papers we have counts for
5 papers · 2 filters
GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4
Tom Kocmi, Christian Federmann
This paper introduces GEMBA-MQM, a GPT-based evaluation metric designed to detect translation quality errors, specifically for the quality estimation setting without the need for h…
SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window
Vikas Raunak, Tom Kocmi, Matt Post
Reference-based metrics that operate at the sentence-level typically outperform quality estimation metrics, which have access only to the source and system output. This is unsurpri…
Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References
Tianyi Tang, Hongyuan Lu, Yuchen Eleanor Jiang +5
Most research about natural language generation (NLG) relies on evaluation benchmarks with limited references for a sample, which may result in poor correlations with human judgeme…
Large Language Models Are State-of-the-Art Evaluators of Translation Quality
Tom Kocmi, Christian Federmann
We describe GEMBA, a GPT-based metric for assessment of translation quality, which works both with a reference translation and without. In our evaluation, we focus on zero-shot pro…
Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou +4
Machine translation quality estimation (QE) predicts human judgements of a translation hypothesis without seeing the reference. State-of-the-art QE systems based on pretrained lang…