106 citations · 785 across the 59 of their papers we have counts for
71 papers
Evaluating Automatic Metrics with Incremental Machine Translation Systems
Guojun Wu, Shay B. Cohen, Rico Sennrich
We introduce a dataset comprising commercial machine translations, gathered weekly over six years across 12 translation directions. Since human A/B testing is commonly used, we ass…
SignCLIP: Connecting Text and Sign Language by Contrastive Learning
Zifan Jiang, Gerard Sant, Amit Moryossef +3
We present SignCLIP, which re-purposes CLIP (Contrastive Language-Image Pretraining) to project spoken language text and sign language videos, two classes of natural languages of d…
Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets
Nikita Moghe, Arnisa Fazla, Chantal Amrhein +5
Recent machine translation (MT) metrics calibrate their effectiveness by correlating with human judgement but without any insights about their behaviour across different error type…
A Benchmark for Evaluating Machine Translation Metrics on Dialects Without Standard Orthography
Noëmi Aepli, Chantal Amrhein, Florian Schottmann +1
For sensible progress in natural language processing, it is important that we are aware of the limitations of the evaluation metrics we use. In this work, we evaluate how robust me…
Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding
Rico Sennrich, Jannis Vamvas, Alireza Mohammadshahi
Hallucinations and off-target translation remain unsolved problems in MT, especially for low-resource languages and massively multilingual models. In this paper, we introduce two r…
Uncertainty in Natural Language Generation: From Theory to Applications
Joris Baan, Nico Daheim, Evgenia Ilia +7
Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like sum…
Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model
Chantal Amrhein, Florian Schottmann, Rico Sennrich +1
Natural language generation models reproduce and often amplify the biases present in their training data. Previous research explored using sequence-to-sequence rewriting models to…
What's the Meaning of Superhuman Performance in Today's NLU?
Simone Tedeschi, Johan Bos, Thierry Declerck +9
In the last five years, there has been a significant focus in Natural Language Processing (NLP) on developing larger Pretrained Language Models (PLMs) and introducing benchmarks su…
SLTUNET: A Simple Unified Model for Sign Language Translation
Biao Zhang, Mathias Müller, Rico Sennrich
Despite recent successes with neural models for sign language translation (SLT), translation quality still lags behind spoken languages because of the data scarcity and modality ga…
Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents
Jannis Vamvas, Rico Sennrich
Automatically highlighting words that cause semantic differences between two documents could be useful for a wide range of applications. We formulate recognizing semantic differenc…
SwissBERT: The Multilingual Language Model for Switzerland
Jannis Vamvas, Johannes Graën, Rico Sennrich
We present SwissBERT, a masked language model created specifically for processing Switzerland-related text. SwissBERT is a pre-trained model that we adapted to news articles writte…
Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation
Biao Zhang, Barry Haddow, Rico Sennrich
For end-to-end speech translation, regularizing the encoder with the Connectionist Temporal Classification (CTC) objective using the source transcript or target translation as labe…
Improving the Cross-Lingual Generalisation in Visual Question Answering
Farhad Nooralahzadeh, Rico Sennrich
While several benefits were realized for multilingual vision-language pretrained models, recent benchmarks across various tasks and languages showed poor cross-lingual generalisati…
A Priority Map for Vision-and-Language Navigation with Trajectory Plans and Feature-Location Cues
Jason Armitage, Leonardo Impett, Rico Sennrich
In a busy city street, a pedestrian surrounded by distractions can pick out a single sign if it is relevant to their route. Artificial agents in outdoor Vision-and-Language Navigat…
Revisiting End-to-End Speech-to-Text Translation From Scratch
Biao Zhang, Barry Haddow, Rico Sennrich
End-to-end (E2E) speech-to-text translation (ST) often depends on pretraining its encoder and/or decoder using source transcripts via speech recognition or text translation tasks,…
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages
Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya +11
Natural language processing (NLP) has a significant impact on society via technologies such as machine translation and search engines. Despite its success, NLP technology is only w…
NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measures
Jannis Vamvas, Rico Sennrich
Being able to rank the similarity of short text segments is an interesting bonus feature of neural machine translation. Translation-based similarity measures include direct and piv…
As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioning
Jannis Vamvas, Rico Sennrich
Omission and addition of content is a typical issue in neural machine translation. We propose a method for detecting such phenomena with off-the-shelf translation models. Using con…
Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMET
Chantal Amrhein, Rico Sennrich
Neural metrics have achieved impressive correlation with human judgements in the evaluation of machine translation systems, but before we can safely optimise towards such metrics,…
Distributionally Robust Recurrent Decoders with Random Network Distillation
Antonio Valerio Miceli-Barone, Alexandra Birch, Rico Sennrich
Neural machine learning models can successfully model language that is similar to their training distribution, but they are highly susceptible to degradation under distribution shi…
On the Limits of Minimal Pairs in Contrastive Evaluation
Jannis Vamvas, Rico Sennrich
Minimal sentence pairs are frequently used to analyze the behavior of language models. It is often assumed that model behavior on contrastive pairs is predictive of model behavior…
Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models
Jiaoda Li, Duygu Ataman, Rico Sennrich
Multimodal machine translation (MMT) systems have been shown to outperform their text-only neural machine translation (NMT) counterparts when visual context is available. However,…
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT
Elena Voita, Rico Sennrich, Ivan Titov
Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural ne…
How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?
Chantal Amrhein, Rico Sennrich
Data-driven subword segmentation has become the default strategy for open-vocabulary machine translation and other NLP tasks, but may not be sufficiently generic for optimal learni…