Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
arXiv:2006.08344
Abstract
We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure of uncertainty designed specifically for long sequences of discrete random variables -- i.e. words in the output sentence. Our new measure of uncertainty solves a major intractability in the naive application of existing approaches on long sentences. We use our new measure on a Transformer model trained with dropout approximate inference. On the task of German-English translation using WMT13 and Europarl, we show that with dropout uncertainty our measure is able to identify when Dutch source sentences, sentences which use the same word types as German, are given to the model instead of German.
arXiv comment19 pages, 9 figures
References in corpus (11)
- A Survey on Deep Learning in Medical Image Analysis
- Neural Machine Translation by Jointly Learning to Align and Translate
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- On Calibration of Modern Neural Networks
- Effective Approaches to Attention-based Neural Machine Translation
- Deep Bayesian Active Learning with Image Data
- Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
- Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
- Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning
- Calibration of Encoder Decoder Models for Neural Machine Translation
- Bayesian Deep Learning for Exoplanet Atmospheric Retrieval