52 citations · 101 across the 8 of their papers we have counts for
14 papers
Rank-One Editing of Encoder-Decoder Models
Vikas Raunak, Arul Menezes
Large sequence to sequence models for tasks such as Neural Machine Translation (NMT) are usually trained over hundreds of millions of samples. However, training is just the origin…
Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models
Vikas Raunak, Matt Post, Arul Menezes
In this work, we present some recommendations on the evaluation of state-of-the-art generative models for constrained generation tasks. The progress on generative models has been r…
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
Vikas Raunak, Arul Menezes
Memorization presents a challenge for several constrained Natural Language Generation (NLG) tasks such as Neural Machine Translation (NMT), wherein the proclivity of neural models…
SALTED: A Framework for SAlient Long-Tail Translation Error Detection
Vikas Raunak, Matt Post, Arul Menezes
Traditional machine translation (MT) metrics provide an average measure of translation quality that is insensitive to the long tail of behavioral problems in MT. Examples include t…
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
Siddharth Dalmia, Brian Yan, Vikas Raunak +2
End-to-end approaches for sequence tasks are becoming increasingly popular. Yet for complex sequence tasks, like speech translation, systems that cascade several models trained on…
The Curious Case of Hallucinations in Neural Machine Translation
Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt
In this work, we study hallucinations in Neural Machine Translation (NMT), which lie at an extreme end on the spectrum of NMT pathologies. Firstly, we connect the phenomenon of hal…