92 citations · 301 across the 26 of their papers we have counts for
33 papers
Speaker Diarization of Scripted Audiovisual Content
Yogesh Virkar, Brian Thompson, Rohit Paturi +2
The media localization industry usually requires a verbatim script of the final film or TV production in order to create subtitles or dubbing scripts in a foreign language. In part…
Improving Isochronous Machine Translation with Target Factors and Auxiliary Counters
Proyag Pal, Brian Thompson, Yogesh Virkar +3
To translate speech for automatic dubbing, machine translation needs to be isochronous, i.e. translated speech needs to be aligned with the source in terms of speech durations. We…
Jointly Optimizing Translations and Speech Timing to Improve Isochrony in Automatic Dubbing
Alexandra Chronopoulou, Brian Thompson, Prashant Mathur +3
Automatic dubbing (AD) is the task of translating the original speech in a video into target language speech. The new target language speech should satisfy isochrony; that is, the…
A baseline revisited: Pushing the limits of multi-segment models for context-aware translation
Suvodeep Majumder, Stanislas Lauly, Maria Nadejde +2
This paper addresses the task of contextual translation using multi-segment models. Specifically we show that increasing model capacity further pushes the limits of this approach a…
Improving Robustness of Retrieval Augmented Translation via Shuffling of Suggestions
Cuong Hoang, Devendra Sachan, Prashant Mathur +2
Several recent studies have reported dramatic performance improvements in neural machine translation (NMT) by augmenting translation at inference time with fuzzy-matches retrieved…
Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions
Cuong Hoang, Devendra Sachan, Prashant Mathur +2
We explore zero-shot adaptation, where a general-domain model has access to customer or domain specific parallel data at inference time, but not during training. We build on the id…
Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metric
Giorgos Vernikos, Brian Thompson, Prashant Mathur +1
We hypothesize that existing sentence-level machine translation (MT) metrics become less effective when the human reference contains ambiguities. To verify this hypothesis, we pres…
Sockeye 3: Fast Neural Machine Translation with PyTorch
Felix Hieber, Michael Denkowski, Tobias Domhan +11
Sockeye 3 is the latest version of the Sockeye toolkit for Neural Machine Translation (NMT). Now based on PyTorch, Sockeye 3 provides faster model implementations and more advanced…
CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality
Maria Nădejde, Anna Currey, Benjamin Hsu +3
The machine translation (MT) task is typically formulated as that of returning a single translation for an input segment. However, in many cases, multiple different translations ar…
Prosodic Alignment for off-screen automatic dubbing
Yogesh Virkar, Marcello Federico, Robert Enyedi +1
The goal of automatic dubbing is to perform speech-to-speech translation while achieving audiovisual coherence. This entails isochrony, i.e., translating the original speech by als…
Isometric MT: Neural Machine Translation for Automatic Dubbing
Surafel M. Lakew, Yogesh Virkar, Prashant Mathur +1
Automatic dubbing (AD) is among the machine translation (MT) use cases where translations should match a given length to allow for synchronicity between source and target speech. F…
Isochrony-Aware Neural Machine Translation for Automatic Dubbing
Derek Tam, Surafel M. Lakew, Yogesh Virkar +2
We introduce the task of isochrony-aware machine translation which aims at generating translations suitable for dubbing. Dubbing of a spoken sentence requires transferring the cont…
Machine Translation Verbosity Control for Automatic Dubbing
Surafel M. Lakew, Marcello Federico, Yue Wang +4
Automatic dubbing aims at seamlessly replacing the speech in a video document with synthetic speech in a different language. The task implies many challenges, one of which is gener…
TICO-19: the Translation Initiative for Covid-19
Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou +15
The COVID-19 pandemic is the worst pandemic to strike the world in over a century. Crucial to stemming the tide of the SARS-CoV-2 virus is communicating to vulnerable populations t…
Joint translation and unit conversion for end-to-end localization
Georgiana Dinu, Prashant Mathur, Marcello Federico +2
A variety of natural language tasks require processing of textual data which contains a mix of natural language and formal languages such as mathematical expressions. In this paper…
From Speech-to-Speech Translation to Automatic Dubbing
Marcello Federico, Robert Enyedi, Roberto Barra-Chicote +4
We present enhancements to a speech-to-speech translation pipeline in order to perform automatic dubbing. Our architecture features neural machine translation generating output of…
Adapting Multilingual Neural Machine Translation to Unseen Languages
Surafel M. Lakew, Alina Karakanta, Marcello Federico +2
Multilingual Neural Machine Translation (MNMT) for low-resource languages (LRL) can be enhanced by the presence of related high-resource languages (HRL), but the relatedness of HRL…
Controlling the Output Length of Neural Machine Translation
Surafel Melaku Lakew, Mattia Di Gangi, Marcello Federico
The recent advances introduced by neural machine translation (NMT) are rapidly expanding the application fields of machine translation, as well as reshaping the quality level to be…
Robust Neural Machine Translation for Clean and Noisy Speech Transcripts
Mattia Antonino Di Gangi, Robert Enyedi, Alessandra Brusadin +1
Neural machine translation models have shown to achieve high quality when trained and fed with well structured and punctuated input texts. Unfortunately, the latter condition is no…
On the Importance of Word Boundaries in Character-level Neural Machine Translation
Duygu Ataman, Orhan Firat, Mattia A. Di Gangi +2
Neural Machine Translation (NMT) models generally perform translation using a fixed-size lexical vocabulary, which is an important bottleneck on their generalization capability and…
Multilingual Neural Machine Translation for Zero-Resource Languages
Surafel M. Lakew, Marcello Federico, Matteo Negri +1
In recent years, Neural Machine Translation (NMT) has been shown to be more effective than phrase-based statistical methods, thus quickly becoming the state of the art in machine t…
Training Neural Machine Translation To Apply Terminology Constraints
Georgiana Dinu, Prashant Mathur, Marcello Federico +1
This paper proposes a novel method to inject custom terminology into neural machine translation at run time. Previous works have mainly proposed modifications to the decoding algor…
Phonetically-Oriented Word Error Alignment for Speech Recognition Error Analysis in Speech Translation
Nicholas Ruiz, Marcello Federico
We propose a variation to the commonly used Word Error Rate (WER) metric for speech recognition evaluation which incorporates the alignment of phonemes, in the absence of time boun…
Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors
Nicholas Ruiz, Mattia Antonino Di Gangi, Nicola Bertoldi +1
Machine translation systems are conventionally trained on textual resources that do not model phenomena that occur in spoken language. While the evaluation of neural machine transl…