Faithful to the Original: Fact Aware Neural Abstractive Summarization

Abstract

Unlike extractive summarization, abstractive summarization has to fuse different parts of the source text, which inclines to create fake facts. Our preliminary study reveals nearly 30% of the outputs from a state-of-the-art neural summarization system suffer from this problem. While previous abstractive summarization approaches usually focus on the improvement of informativeness, we argue that faithfulness is also a vital prerequisite for a practical abstractive summarization system. To avoid generating fake facts in a summary, we leverage open information extraction and dependency parse technologies to extract actual fact descriptions from the source text. The dual-attention sequence-to-sequence framework is then proposed to force the generation conditioned on both the source text and the extracted fact descriptions. Experiments on the Gigaword benchmark dataset demonstrate that our model can greatly reduce fake summaries by 80%. Notably, the fact descriptions also bring significant improvement on informativeness since they often condense the meaning of the source text.

8 pages, 3 figures, AAAI 2018

Cited by in corpus (35)

• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
• Neural Language Generation: Formulation, Methods, and Evaluation
• BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization
• Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward
• On Hallucination and Predictive Uncertainty in Conditional Language Generation
• Alleviating Exposure Bias via Contrastive Learning for Abstractive Text Summarization
• CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarization
• Scoring Sentence Singletons and Pairs for Abstractive Summarization
• Reducing Quantity Hallucinations in Abstractive Summarization
• Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs
• Hooks in the Headline: Learning to Generate Headlines with Controlled Styles
• TODSum: Task-Oriented Dialogue Summarization with State Tracking
• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization
• Improving Question Generation With to the Point Context
• Multi-Fact Correction in Abstractive Text Summarization
• Entity-level Factual Consistency of Abstractive Text Summarization
• Text Generation with Exemplar-based Adaptive Decoding
• Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization
• Analyzing Sentence Fusion in Abstractive Summarization
• Contrastive Attention Mechanism for Abstractive Sentence Summarization
• Guiding Extractive Summarization with Question-Answering Rewards
• BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization
• Improving Factual Consistency of Abstractive Summarization on Customer Feedback
• Fact-level Extractive Summarization with Hierarchical Graph Mask on BERT
• A New Approach to Overgenerating and Scoring Abstractive Summaries
• Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features
• Truth-Conditional Captioning of Time Series Data
• Joint Parsing and Generation for Abstractive Summarization
• Improving Truthfulness of Headline Generation
• Summary Explorer: Visualizing the State of the Art in Text Summarization
• MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization
• CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
• StreamHover: Livestream Transcript Summarization and Annotation
• AGGGEN: Ordering and Aggregating while Generating