Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond

Abstract

In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora. We propose several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling key-words, capturing the hierarchy of sentence-to-word structure, and emitting words that are rare or unseen at training time. Our work shows that many of our proposed models contribute to further improvement in performance. We also propose a new dataset consisting of multi-sentence summaries, and establish performance benchmarks for further research.

References in corpus (6)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• ADADELTA: An Adaptive Learning Rate Method
• A Hierarchical Neural Autoencoder for Paragraphs and Documents
• Neural Summarization by Extracting Sentences and Words
• Pointing the Unknown Words
• On Using Very Large Target Vocabulary for Neural Machine Translation

Cited by in corpus (149)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• A Deep Reinforced Model for Abstractive Summarization
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Selective Encoding for Abstractive Sentence Summarization
• Online and Linear-Time Attention by Enforcing Monotonic Alignments
• A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)
• On the Dimensionality of Word Embedding
• Abstract Meaning Representation for Multi-Document Summarization
• ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
• OpenNMT: Neural Machine Translation Toolkit
• Efficient Summarization with Read-Again and Copy Mechanism
• Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
• Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models
• Topic-Guided Variational Autoencoders for Text Generation
• HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
• Learning to Decode for Future Success
• Graph-based Neural Multi-Document Summarization
• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• Actor-Critic based Training Framework for Abstractive Summarization
• The price of debiasing automatic metrics in natural language evaluation
• Neural Models for Sequence Chunking
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Incorporating BERT into Parallel Sequence Decoding with Adapters
• Unifying Human and Statistical Evaluation for Natural Language Generation
• Recent advances in conversational NLP : Towards the standardization of Chatbot building
• Deep Learning Based Chatbot Models
• Generate Natural Language Explanations for Recommendation
• Abstractive and Extractive Text Summarization using Document Context Vector and Recurrent Neural Networks
• Neural Question Generation from Text: A Preliminary Study
• LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5
• Generating Sentences by Editing Prototypes
• Deep Recurrent Generative Decoder for Abstractive Text Summarization
• Data-driven Summarization of Scientific Articles
• A multi-task learning model for malware classification with useful file access pattern from API call sequence
• Semi-Supervised Speech Recognition via Local Prior Matching
• A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization
• Controllable Abstractive Summarization
• Focused Hierarchical RNNs for Conditional Sequence Processing
• QuestEval: Summarization Asks for Fact-based Evaluation
• On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
• Neural Language Generation: Formulation, Methods, and Evaluation
• Paraphrase Generation with Deep Reinforcement Learning
• Extractive Summarization of EHR Discharge Notes
• Knowledge driven Description Synthesis for Floor Plan Interpretation
• Deep Learning Model on Gravitational Waveforms in Merging and Ringdown Phases of Binary Black Hole Coalescences
• Strategies for Structuring Story Generation
• Unsupervised Pretraining for Sequence to Sequence Learning
• Neural Document Summarization by Jointly Learning to Score and Select Sentences
• Dial2Desc: End-to-end Dialogue Description Generation
• A Condense-then-Select Strategy for Text Summarization
• RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization
• One-Shot Learning for Text-to-SQL Generation
• What comes next? Extractive summarization by next-sentence prediction
• AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
• In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
• A Baseline Analysis for Podcast Abstractive Summarization
• Entity Commonsense Representation for Neural Abstractive Summarization
• Language Style Transfer from Sentences with Arbitrary Unknown Styles
• Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?
• Improving Neural Question Generation using Answer Separation
• Discriminative Adversarial Search for Abstractive Summarization
• Understanding Multi-Head Attention in Abstractive Summarization
• Neural Latent Extractive Document Summarization
• Learning to Summarize Radiology Findings
• Learning to update Auto-associative Memory in Recurrent Neural Networks for Improving Sequence Memorization
• Video Highlight Prediction Using Audience Chat Reactions
• Improved Natural Language Generation via Loss Truncation
• Enhancing Scientific Papers Summarization with Citation Graph
• Attention based end to end Speech Recognition for Voice Search in Hindi and English
• An Entity-Driven Framework for Abstractive Summarization
• Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
• Hooks in the Headline: Learning to Generate Headlines with Controlled Styles
• Point-less: More Abstractive Summarization with Pointer-Generator Networks
• Toward Extractive Summarization of Online Forum Discussions via Hierarchical Attention Networks
• Learning to Attend, Copy, and Generate for Session-Based Query Suggestion
• DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
• Conditional Text Generation for Harmonious Human-Machine Interaction
• Dual Recurrent Attention Units for Visual Question Answering
• Automatic Generation of Chinese Short Product Titles for Mobile Display
• Controllable Abstractive Dialogue Summarization with Sketch Supervision
• Abstractive Summarization for Low Resource Data using Domain Transfer and Data Synthesis
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• ColdGANs: Taming Language GANs with Cautious Sampling Strategies
• Teaching Machines to Converse
• Why Do Neural Response Generation Models Prefer Universal Replies?
• Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots
• Contextualizing ASR Lattice Rescoring with Hybrid Pointer Network Language Model
• Arabic aspect based sentiment analysis using bidirectional GRU based models
• Outline Generation: Understanding the Inherent Content Structure of Documents
• CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
• SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation
• Subjective Bias in Abstractive Summarization
• FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation
• Efficient and Effective Single-Document Summarizations and A Word-Embedding Measurement of Quality
• Neural Linguistic Steganography
• Combining Word Embeddings and N-grams for Unsupervised Document Summarization
• Closed-Book Training to Improve Summarization Encoder Memory
• Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning
• Positional Attention-based Frame Identification with BERT: A Deep Learning Approach to Target Disambiguation and Semantic Frame Selection
• BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation
• PoinT-5: Pointer Network and T-5 based Financial NarrativeSummarisation
• Improving generation quality of pointer networks via guided attention
• VT-SSum: A Benchmark Dataset for Video Transcript Segmentation and Summarization
• A more abstractive summarization model
• On the impressive performance of randomly weighted encoders in summarization tasks
• Latent Semantic Analysis Approach for Document Summarization Based on Word Embeddings
• Salience Estimation with Multi-Attention Learning for Abstractive Text Summarization
• LSICC: A Large Scale Informal Chinese Corpus
• Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
• Simple Unsupervised Summarization by Contextual Matching
• Contextualized Rewriting for Text Summarization
• AIBench Training: Balanced Industry-Standard AI Training Benchmarking
• Self-organized Hierarchical Softmax
• SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline
• Relevance-Promoting Language Model for Short-Text Conversation
• A Hybrid Word-Character Approach to Abstractive Summarization
• Long Short-Term Memory with Dynamic Skip Connections
• Controllable Length Control Neural Encoder-Decoder via Reinforcement Learning
• Live Blog Corpus for Summarization
• Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System
• Pointing to Subwords for Generating Function Names in Source Code
• Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization
• Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization
• Learning Sentence Embeddings for Coherence Modelling and Beyond
• An analysis of document graph construction methods for AMR summarization
• Describing Semantic Representations of Brain Activity Evoked by Visual Stimuli
• DeepTitle -- Leveraging BERT to generate Search Engine Optimized Headlines
• Language Transfer of Audio Word2Vec: Learning Audio Segment Representations without Target Language Data
• News Meets Microblog: Hashtag Annotation via Retriever-Generator
• Eliciting Knowledge from Experts:Automatic Transcript Parsing for Cognitive Task Analysis
• Exploring a Unified Sequence-To-Sequence Transformer for Medical Product Safety Monitoring in Social Media
• Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation
• Multiple Generative Models Ensemble for Knowledge-Driven Proactive Human-Computer Dialogue Agent
• Dialogue Summarization with Supporting Utterance Flow Modeling and Fact Regularization
• AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions
• Centrality Meets Centroid: A Graph-based Approach for Unsupervised Document Summarization
• INS: An Interactive Chinese News Synthesis System
• Ranking sentences from product description & bullets for better search
• Saliency Maps Generation for Automatic Text Summarization
• From web crawled text to project descriptions: automatic summarizing of social innovation projects
• Evaluating Document Coherence Modelling
• Improving Abstractive Text Summarization with History Aggregation
• A Spark ML driven preprocessing approach for deep learning based scholarly data applications
• Cross Copy Network for Dialogue Generation
• Extractive Research Slide Generation Using Windowed Labeling Ranking
• Abstractive Summarization Improved by WordNet-based Extractive Sentences
• Hierarchical GPT with Congruent Transformers for Multi-Sentence Language Models
• AI-lead Court Debate Case Investigation