Publications (11)
On Biasing Transformer Attention Towards Monotonicity
Annette Rios, Chantal Amrhein, Noëmi Aepli +1
Many sequence-to-sequence tasks in natural language processing are roughly monotonic in the alignment between source and target sequence, and previous work has facilitated or enfor…
Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures
Gongbo Tang, Mathias Müller, Annette Rios +1
Recently, non-recurrent architectures (convolutional, self-attentional) have outperformed RNNs in neural machine translation. CNNs and self-attentional networks can connect distant…
A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation
Mathias Müller, Annette Rios, Elena Voita +1
The translation of pronouns presents a special challenge to machine translation to this day, since it often requires context outside the current sentence. Recent work on models tha…
Meaningful Pose-Based Sign Language Evaluation
Zifan Jiang, Colin Leong, Amit Moryossef +8
We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based,…
Considerations for meaningful sign language machine translation based on glosses
Mathias Müller, Zifan Jiang, Amit Moryossef +2
Automatic sign language processing is gaining popularity in Natural Language Processing (NLP) research (Yin et al., 2021). In machine translation (MT) in particular, sign language…
German also Hallucinates! Inconsistency Detection in News Summaries with the Absinth Dataset
Laura Mascarell, Ribin Chalumattu, Annette Rios
The advent of Large Language Models (LLMs) has led to remarkable progress on a wide range of natural language processing tasks. Despite the advances, these large-sized models still…
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Julia Kreutzer, Isaac Caswell, Lisa Wang +49
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text dataset…
Domain Robustness in Neural Machine Translation
Mathias Müller, Annette Rios, Rico Sennrich
Translating text that diverges from the training domain is a key challenge for machine translation. Domain robustness---the generalization of models to unseen test domains---is low…
Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition
Amit Moryossef, Ioannis Tsochantaridis, Joe Dinn +6
Signed languages are visual languages produced by the movement of the hands, face, and body. In this paper, we evaluate representations based on skeleton poses, as these are explai…
Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translation
Annette Rios, Mathias Müller, Rico Sennrich
Zero-shot neural machine translation is an attractive goal because of the high cost of obtaining data and building translation systems for new translation directions. However, prev…
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages
Abteen Ebrahimi, Manuel Mager, Arturo Oncevay +14
Pretrained multilingual models are able to perform cross-lingual transfer in a zero-shot setting, even for languages unseen during pretraining. However, prior work evaluating perfo…