Detecting Hallucinated Content in Conditional Neural Sequence Generation

Abstract

Neural sequence models can generate highly fluent sentences, but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input. These variety of fluent but wrong outputs are particularly problematic, as it will not be possible for users to tell they are being presented incorrect content. To detect these errors, we propose a task to predict whether each token in the output sequence is hallucinated (not contained in the input) and collect new manually annotated evaluation sets for this task. We also introduce a method for learning to detect hallucinations using pretrained language models fine tuned on synthetic data that includes automatically inserted hallucinations Experiments on machine translation (MT) and abstractive summarization demonstrate that our proposed approach consistently outperforms strong baselines on all benchmark datasets. We further demonstrate how to use the token-level hallucination labels to define a fine-grained loss over the target sequence in low-resource MT and achieve significant improvements over strong baseline methods. We also apply our method to word-level quality estimation for MT and show its effectiveness in both supervised and unsupervised settings. Codes and data available at https://github.com/violet-zct/fairseq-detect-hallucination.

arXiv commentAccepted by ACL-Finding 2021

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Language Models are Few-Shot Learners
• Convolutional Sequence to Sequence Learning
• BERTScore: Evaluating Text Generation with BERT
• Longformer: The Long-Document Transformer
• Multilingual Denoising Pre-training for Neural Machine Translation
• Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
• Sequence-Level Knowledge Distillation
• PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
• Domain Robustness in Neural Machine Translation
• Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation
• ELI5: Long Form Question Answering
• Parallel Corpus Filtering via Pre-trained Language Models

Cited by in corpus (8)

• The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey
• Attention-guided Generative Models for Extractive Question Answering
• Mitigating Media Bias through Neutral Article Generation
• Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation
• AgreeSum: Agreement-Oriented Multi-Document Summarization
• Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation
• Miðeind's WMT 2021 submission
• Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation