6 citations · 22 across the 10 of their papers we have counts for
11 papers
Looking under the Wrong Lamppost: On the Limitations of Automated Translation Quality Estimation
Serge Gladkoff, Angelika Vaasa, Sue Ellen Wright +2
Automation of Translation Quality Estimation (QE) has emerged as a widely discussed approach to managing translation quality at scale, and a growing number of tools and technologie…
Non-Linear Scoring Model for Translation Quality Evaluation
Serge Gladkoff, Lifeng Han, Katerina Gasova
Analytic Translation Quality Evaluation (TQE), based on Multidimensional Quality Metrics (MQM), traditionally uses a linear error-to-penalty scale calibrated to a reference sample…
The Multi-Range Theory of Translation Quality Measurement: MQM scoring models and Statistical Quality Control
Arle Lommel, Serge Gladkoff, Alan Melby +10
The year 2024 marks the 10th anniversary of the Multidimensional Quality Metrics (MQM) framework for analytic translation quality evaluation. The MQM error typology has been widely…
Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-Learning
Lifeng Han, Serge Gladkoff, Gleb Erofeev +3
We conduct investigations on clinical text machine translation by examining multilingual neural network models using deep learning such as Transformer based structures. Furthermore…
MTUncertainty: Assessing the Need for Post-editing of Machine Translation Outputs by Fine-tuning OpenAI LLMs
Serge Gladkoff, Lifeng Han, Gleb Erofeev +2
Translation Quality Evaluation (TQE) is an essential step of the modern translation production process. TQE is critical in assessing both machine translation (MT) and human transla…
Student's t-Distribution: On Measuring the Inter-Rater Reliability When the Observations are Scarce
Serge Gladkoff, Lifeng Han, Goran Nenadic
In natural language processing (NLP) we always rely on human judgement as the golden quality evaluation method. However, there has been an ongoing debate on how to better evaluate…
Examining Large Pre-Trained Language Models for Machine Translation: What You Don't Know About It
Lifeng Han, Gleb Erofeev, Irina Sorokina +2
Pre-trained language models (PLMs) often take advantage of the monolingual and multilingual dataset that is freely available online to acquire general or mixed domain knowledge bef…
Investigating Massive Multilingual Pre-Trained Machine Translation Models for Clinical Domain via Transfer Learning
Lifeng Han, Gleb Erofeev, Irina Sorokina +2
Massively multilingual pre-trained language models (MMPLMs) are developed in recent years demonstrating superpowers and the pre-knowledge they acquire for downstream tasks. This wo…
HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation
Serge Gladkoff, Lifeng Han
Traditional automatic evaluation metrics for machine translation have been widely criticized by linguists due to their low accuracy, lack of transparency, focus on language mechani…
Measuring Uncertainty in Translation Quality Evaluation (TQE)
Serge Gladkoff, Irina Sorokina, Lifeng Han +1
From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task. Translation service providers (…
cushLEPOR: customising hLEPOR metric using Optuna for higher agreement with human judgments or pre-trained language model LaBSE
Lifeng Han, Irina Sorokina, Gleb Erofeev +1
Human evaluation has always been expensive while researchers struggle to trust the automatic metrics. To address this, we propose to customise traditional metrics by taking advanta…