6 citations · 13 across the 16 of their papers we have counts for
17 papers · 1 filter
Déjà Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation
Julia Kreutzer, Eleftheria Briakou, Sweta Agrawal +2
Generation capabilities and language coverage of multilingual large language models (mLLMs) are advancing rapidly. However, evaluation practices for generative abilities of mLLMs a…
On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation
Eleftheria Briakou, Zhongtao Liu, Colin Cherry +1
This paper investigates the impact of verbose LLM translations on evaluation. We first demonstrate the prevalence of this behavior across several LLM outputs drawn from the WMT 202…
Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts
Eleftheria Briakou, Jiaming Luo, Colin Cherry +1
In this paper we present a step-by-step approach to long-form text translation, drawing on established processes in translation studies. Instead of viewing machine translation as a…
Don't Throw Away Data: Better Sequence Knowledge Distillation
Jun Wang, Eleftheria Briakou, Hamid Dadkhahi +3
A critical component in knowledge distillation is the means of coupling the teacher and student. The predominant sequence knowledge distillation method involves supervised learning…
Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differences
Eleftheria Briakou, Navita Goyal, Marine Carpuat
Explainable NLP techniques primarily explain by answering "Which tokens in the input are responsible for this prediction?''. We argue that for NLP models that make predictions by c…
AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages
Jiayi Wang, David Ifeoluwa Adelani, Sweta Agrawal +55
Despite the recent progress on scaling multilingual machine translation (MT) to several under-resourced African languages, accurately measuring this progress remains challenging, s…
Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capability
Eleftheria Briakou, Colin Cherry, George Foster
Large, multilingual language models exhibit surprisingly good zero- or few-shot machine translation capabilities, despite having never seen the intentionally-included translation e…
What Else Do I Need to Know? The Effect of Background Information on Users' Reliance on QA Systems
Navita Goyal, Eleftheria Briakou, Amanda Liu +6
NLP systems have shown impressive performance at answering questions by retrieving relevant context. However, with the increasingly large models, it is impossible and often undesir…
Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection
Weijia Xu, Sweta Agrawal, Eleftheria Briakou +2
Neural sequence generation models are known to "hallucinate", by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remai…
Can Synthetic Translations Improve Bitext Quality?
Eleftheria Briakou, Marine Carpuat
Synthetic translations have been used for a wide range of NLP tasks primarily as a means of data augmentation. This work explores, instead, how synthetic translations can be used t…
BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translation
Eleftheria Briakou, Sida I. Wang, Luke Zettlemoyer +1
Mined bitexts can contain imperfect translations that yield unreliable training signals for Neural Machine Translation (NMT). While filtering such pairs out is known to improve fin…
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer
Eleftheria Briakou, Sweta Agrawal, Joel Tetreault +1
While the field of style transfer (ST) has been growing rapidly, it has been hampered by a lack of standardized practices for automatic evaluation. In this paper, we evaluate leadi…
A Review of Human Evaluation for Style Transfer
Eleftheria Briakou, Sweta Agrawal, Ke Zhang +2
This paper reviews and summarizes human evaluation practices described in 97 style transfer papers with respect to three main evaluation aspects: style transfer, meaning preservati…
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation
Eleftheria Briakou, Marine Carpuat
While it has been shown that Neural Machine Translation (NMT) is highly sensitive to noisy parallel training samples, prior work treats all types of mismatches between source and t…
XFORMAL: A Benchmark for Multilingual Formality Style Transfer
Eleftheria Briakou, Di Lu, Ke Zhang +1
We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese…
Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank
Eleftheria Briakou, Marine Carpuat
Detecting fine-grained differences in content conveyed in different languages matters for cross-lingual NLP and multilingual corpora analysis, but it is a challenging machine learn…
Cross-topic distributional semantic representations via unsupervised mappings
Eleftheria Briakou, Nikos Athanasiou, Alexandros Potamianos
In traditional Distributional Semantic Models (DSMs) the multiple senses of a polysemous word are conflated into a single vector space representation. In this work, we propose a DS…