6 citations · 22 across the 10 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.CL2021★ 4 cited
HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation
Serge Gladkoff, Lifeng Han
Traditional automatic evaluation metrics for machine translation have been widely criticized by linguists due to their low accuracy, lack of transparency, focus on language mechani…
cs.CL2021
Measuring Uncertainty in Translation Quality Evaluation (TQE)
Serge Gladkoff, Irina Sorokina, Lifeng Han +1
From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task. Translation service providers (…
cs.CL2021★ 3 cited
cushLEPOR: customising hLEPOR metric using Optuna for higher agreement with human judgments or pre-trained language model LaBSE
Lifeng Han, Irina Sorokina, Gleb Erofeev +1
Human evaluation has always been expensive while researchers struggle to trust the automatic metrics. To address this, we propose to customise traditional metrics by taking advanta…