Faithful to the Original: Fact Aware Neural Abstractive Summarization

Abstract

Unlike extractive summarization, abstractive summarization has to fuse different parts of the source text, which inclines to create fake facts. Our preliminary study reveals nearly 30% of the outputs from a state-of-the-art neural summarization system suffer from this problem. While previous abstractive summarization approaches usually focus on the improvement of informativeness, we argue that faithfulness is also a vital prerequisite for a practical abstractive summarization system. To avoid generating fake facts in a summary, we leverage open information extraction and dependency parse technologies to extract actual fact descriptions from the source text. The dual-attention sequence-to-sequence framework is then proposed to force the generation conditioned on both the source text and the extracted fact descriptions. Experiments on the Gigaword benchmark dataset demonstrate that our model can greatly reduce fake summaries by 80%. Notably, the fact descriptions also bring significant improvement on informativeness since they often condense the meaning of the source text.

arXiv comment8 pages, 3 figures, AAAI 2018

Cited by in corpus (63)

• Abstract Meaning Representation for Multi-Document Summarization
• Structure-Infused Copy Mechanisms for Abstractive Summarization
• The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
• Neural Extractive Text Summarization with Syntactic Compression
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
• Summarization, Simplification, and Generation: The Case of Patents
• Neural Language Generation: Formulation, Methods, and Evaluation
• Discourse-Aware Neural Extractive Text Summarization
• BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization
• Distilling Knowledge Learned in BERT for Text Generation
• GO FIGURE: A Meta Evaluation of Factuality in Summarization
• Robust Neural Abstractive Summarization Systems and Evaluation against Adversarial Information
• Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward
• On Hallucination and Predictive Uncertainty in Conditional Language Generation
• Alleviating Exposure Bias via Contrastive Learning for Abstractive Text Summarization
• Enhancing Factual Consistency of Abstractive Summarization
• Factual Error Correction for Abstractive Summarization Models
• CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarization
• Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs
• Scoring Sentence Singletons and Pairs for Abstractive Summarization
• Reducing Quantity Hallucinations in Abstractive Summarization
• An Entity-Driven Framework for Abstractive Summarization
• TODSum: Task-Oriented Dialogue Summarization with State Tracking
• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization
• Hooks in the Headline: Learning to Generate Headlines with Controlled Styles
• Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
• Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks
• Reinforced Extractive Summarization with Question-Focused Rewards
• Multi-Task Learning for Cross-Lingual Abstractive Summarization
• Improving Question Generation With to the Point Context
• CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
• Multi-Fact Correction in Abstractive Text Summarization
• Denoising based Sequence-to-Sequence Pre-training for Text Generation
• Improving Factual Consistency of Abstractive Summarization via Question Answering
• Entity-level Factual Consistency of Abstractive Text Summarization
• Analyzing Sentence Fusion in Abstractive Summarization
• Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization
• A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation
• Text Generation with Exemplar-based Adaptive Decoding
• Contrastive Attention Mechanism for Abstractive Sentence Summarization
• Incorporating Question Answering-Based Signals into Abstractive Summarization via Salient Span Selection
• Controllable Length Control Neural Encoder-Decoder via Reinforcement Learning
• BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization
• Guiding Extractive Summarization with Question-Answering Rewards
• Improving Factual Consistency of Abstractive Summarization on Customer Feedback
• Better Highlighting: Creating Sub-Sentence Summary Highlights
• A New Approach to Overgenerating and Scoring Abstractive Summaries
• Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features
• Understanding Neural Abstractive Summarization Models via Uncertainty
• Toward Improving Coherence and Diversity of Slogan Generation
• $Q^2$: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering
• Fact-level Extractive Summarization with Hierarchical Graph Mask on BERT
• A Novel ILP Framework for Summarizing Content with High Lexical Variety
• Summary Explorer: Visualizing the State of the Art in Text Summarization
• CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
• StreamHover: Livestream Transcript Summarization and Annotation
• Truth-Conditional Captioning of Time Series Data
• AGGGEN: Ordering and Aggregating while Generating
• Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation
• Improving Truthfulness of Headline Generation
• Joint Parsing and Generation for Abstractive Summarization
• MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization