7 papers
Beyond Scalar Scores: Reinforcement Learning for Error-Aware Quality Estimation of Machine Translation
Archchana Sindhujan, Girish A. Koushik, Shenbin Qian +2
Quality Estimation (QE) aims to assess the quality of machine translation (MT) outputs without relying on reference translations, making it essential for real-world, large-scale MT…
Automatically Generating Chinese Homophone Words to Probe Machine Translation Estimation Systems
Shenbin Qian, Constantin Orăsan, Diptesh Kanojia +1
Evaluating machine translation (MT) of user-generated content (UGC) involves unique challenges such as checking whether the nuance of emotions from the source are preserved in the…
What do Large Language Models Need for Machine Translation Evaluation?
Shenbin Qian, Archchana Sindhujan, Minnie Kabra +4
Leveraging large language models (LLMs) for various natural language processing tasks has led to superlative claims about their performance. For the evaluation of machine translati…
Are Large Language Models State-of-the-art Quality Estimators for Machine Translation of User-generated Content?
Shenbin Qian, Constantin Orăsan, Diptesh Kanojia +1
This paper investigates whether large language models (LLMs) are state-of-the-art quality estimators for machine translation of user-generated content (UGC) that contains emotional…
A Multi-task Learning Framework for Evaluating Machine Translation of Emotion-loaded User-generated Content
Shenbin Qian, Constantin Orăsan, Diptesh Kanojia +1
Machine translation (MT) of user-generated content (UGC) poses unique challenges, including handling slang, emotion, and literary devices like irony and sarcasm. Evaluating the qua…
An Ensemble Approach to Acronym Extraction using Transformers
Prashant Sharma, Hadeel Saadany, Leonardo Zilio +2
Acronyms are abbreviated units of a phrase constructed by using initial components of the phrase in a text. Automatic extraction of acronyms from a text can help various Natural La…
Pushing the Right Buttons: Adversarial Evaluation of Quality Estimation
Diptesh Kanojia, Marina Fomicheva, Tharindu Ranasinghe +3
Current Machine Translation (MT) systems achieve very good results on a growing variety of language pairs and datasets. However, they are known to produce fluent translation output…