UniTE: Unified Translation Evaluation

Abstract

Translation quality evaluation plays a crucial role in machine translation. According to the input format, it is mainly separated into three tasks, i.e., reference-only, source-only and source-reference-combined. Recent methods, despite their promising results, are specifically designed and optimized on one of them. This limits the convenience of these methods, and overlooks the commonalities among tasks. In this paper, we propose UniTE, which is the first unified framework engaged with abilities to handle all three evaluation tasks. Concretely, we propose monotonic regional attention to control the interaction among input segments, and unified pretraining to better adapt multi-task learning. We testify our framework on WMT 2019 Metrics and WMT 2020 Quality Estimation benchmarks. Extensive analyses show that our \textit{single model} can universally surpass various state-of-the-art or winner methods across tasks. Both source code and associated models are available at https://github.com/NLP2CT/UniTE.

ACL2022

References in corpus (6)

• BARTScore: Evaluating Generated Text as Text Generation
• Towards Understanding Knowledge Distillation
• To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation
• Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
• Intelligent Translation Memory Matching and Retrieval with Sentence Encoders
• RoBLEURT Submission for the WMT2021 Metrics Task

Cited by in corpus (2)

• USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation
• From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation