Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization

Abstract

We consider the problem of modeling the content structure of texts within a specific domain, in terms of the topics the texts address and the order in which these topics appear. We first present an effective knowledge-lean method for learning content models from un-annotated documents, utilizing a novel adaptation of algorithms for Hidden Markov Models. We then apply our method to two complementary tasks: information ordering and extractive summarization. Our experiments show that incorporating content models in these applications yields substantial improvement over previously-proposed methods.

Best paper award

References in corpus (2)

• Multi-Paragraph Segmentation of Expository Text
• Text Segmentation Using Exponential Models

Cited by in corpus (22)

• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• Generating Natural Language Descriptions from OWL Ontologies: the NaturalOWL System
• Probabilistic Frame Induction
• Topic Segmentation and Labeling in Asynchronous Conversations
• Content Modeling Using Latent Permutations
• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
• BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization
• Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward
• Feature Weight Tuning for Recursive Neural Networks
• An Entity-Driven Framework for Abstractive Summarization
• Neural Sentence Ordering Based on Constraint Graphs
• Local and Global Context-Based Pairwise Models for Sentence Ordering
• Evaluating Text Coherence at Sentence and Paragraph Levels
• Sometimes We Want Translationese
• An entity-driven recursive neural network model for chinese discourse coherence modeling
• A Unified Neural Coherence Model
• BERT4SO: Neural Sentence Ordering by Fine-tuning BERT
• Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
• STaCK: Sentence Ordering with Temporal Commonsense Knowledge
• A New Sentence Ordering Method Using BERT Pretrained Model
• Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings
• Stochastic Natural Language Generation Using Dependency Information