105 citations · 231 across the 13 of their papers we have counts for
14 papers · 1 filter
Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracy
Brian Thompson, Nitika Mathur, Daniel Deutsch +1
Selecting an automatic metric that best emulates human annotators is often non-trivial, because there is no clear definition of "best emulates." A meta-metric is required to compar…
Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?
HyoJung Han, Akiko Eriguchi, Haoran Xu +3
Vocabulary adaptation, which integrates new vocabulary into pre-trained language models, enables expansion to new languages and mitigates token over-fragmentation. However, existin…
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale
Haoran Xu, Kenton Murray, Philipp Koehn +3
Large language models (LLMs) have achieved remarkable success across various NLP tasks with a focus on English due to English-centric pre-training and limited multilingual data. In…
On-the-Fly Fusion of Large Language Models and Machine Translation
Hieu Hoang, Huda Khayrallah, Marcin Junczys-Dowmunt
We propose the on-the-fly ensembling of a machine translation model with an LLM, prompted on the same task and input. We perform experiments on 4 language pairs (both directions) w…
SOTASTREAM: A Streaming Approach to Machine Translation Training
Matt Post, Thamme Gowda, Roman Grundkiewicz +3
Many machine translation toolkits make use of a data preparation step wherein raw data is transformed into a tensor format that can be used directly by the trainer. This preparatio…
How to Choose How to Choose Your Chatbot: A Massively Multi-System MultiReference Data Set for Dialog Metric Evaluation
Huda Khayrallah, Zuhaib Akhtar, Edward Cohen +2
We release MMSMR, a Massively Multi-System MultiReference dataset to enable future work on metrics and evaluation for dialog. Automatic metrics for dialogue evaluation should be ro…
Doubly-Trained Adversarial Data Augmentation for Neural Machine Translation
Weiting Tan, Shuoyang Ding, Huda Khayrallah +1
Neural Machine Translation (NMT) models are known to suffer from noisy inputs. To make models robust, we generate adversarial augmentation samples that attack the model and preserv…
SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training
Huda Khayrallah, João Sedoc
Non-task-oriented dialog models suffer from poor quality and non-diverse responses. To overcome limited conversational data, we apply Simulated Multiple Reference Training (SMRT; K…
Measuring the `I don't know' Problem through the Lens of Gricean Quantity
Huda Khayrallah, João Sedoc
We consider the intrinsic evaluation of neural generative dialog models through the lens of Grice's Maxims of Conversation (1975). Based on the maxim of Quantity (be informative),…
Simulated Multiple Reference Training Improves Low-Resource Machine Translation
Huda Khayrallah, Brian Thompson, Matt Post +1
Many valid translations exist for a given sentence, yet machine translation (MT) is trained with a single reference translation, exacerbating data sparsity in low-resource settings…
An Empirical Exploration of Curriculum Learning for Neural Machine Translation
Xuan Zhang, Gaurav Kumar, Huda Khayrallah +6
Machine translation systems based on deep neural networks are expensive to train. Curriculum learning aims to address this issue by choosing the order in which samples are presente…
Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
Brian Thompson, Huda Khayrallah, Antonios Anastasopoulos +7
To better understand the effectiveness of continued training, we analyze the major components of a neural machine translation system (the encoder, decoder, and each embedding space…
On the Impact of Various Types of Noise on Neural Machine Translation
Huda Khayrallah, Philipp Koehn
We examine how various types of noise in the parallel training data impact the quality of neural machine translation systems. We create five types of artificial noise and analyze h…
Paradigm Completion for Derivational Morphology
Ryan Cotterell, Ekaterina Vylomova, Huda Khayrallah +2
The generation of complex derived word forms has been an overlooked problem in NLP; we fill this gap by applying neural sequence-to-sequence models to the task. We overview the the…