On Hallucination and Predictive Uncertainty in Conditional Language Generation

Abstract

Despite improvements in performances on different natural language generation tasks, deep neural models are prone to hallucinating facts that are incorrect or nonexistent. Different hypotheses are proposed and examined separately for different tasks, but no systematic explanations are available across these tasks. In this study, we draw connections between hallucinations and predictive uncertainty in conditional language generation. We investigate their relationship in both image captioning and data-to-text generation and propose a simple extension to beam search to reduce hallucination. Our analysis shows that higher predictive uncertainty corresponds to a higher chance of hallucination. Epistemic uncertainty is more indicative of hallucination than aleatoric or total uncertainties. It helps to achieve better results of trading performance in standard metric for less hallucination with the proposed beam search variant.

EACL 2021

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Weight Uncertainty in Neural Networks
• Deep and Confident Prediction for Time Series at Uber
• Faithful to the Original: Fact Aware Neural Abstractive Summarization
• Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
• Discriminability objective for training descriptive captions
• Domain Robustness in Neural Machine Translation
• Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation