Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
arXiv:1811.00783
Abstract
We address the problem of abstractive summarization in two directions: proposing a novel dataset and a new model. First, we collect Reddit TIFU dataset, consisting of 120K posts from the online discussion forum Reddit. We use such informal crowd-generated posts as text source, in contrast with existing datasets that mostly use formal documents as source such as news articles. Thus, our dataset could less suffer from some biases that key sentences usually locate at the beginning of the text and favorable summary candidates are already inside the text in similar forms. Second, we propose a novel abstractive summarization model named multi-level memory networks (MMN), equipped with multi-level memory to store the information of text from different levels of abstraction. With quantitative evaluation and user studies via Amazon Mechanical Turk, we show the Reddit TIFU dataset is highly abstractive and the MMN outperforms the state-of-the-art summarization models.
arXiv commentPublished in NAACL-HLT 2019 (Oral)
References in corpus (12)
- Sequence to Sequence Learning with Neural Networks
- WaveNet: A Generative Model for Raw Audio
- Teaching Machines to Read and Comprehend
- Convolutional Sequence to Sequence Learning
- Conditional Image Generation with PixelCNN Decoders
- A Deep Reinforced Model for Abstractive Summarization
- SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
- End-To-End Memory Networks
- Layer Normalization
- Regularizing Neural Networks by Penalizing Confident Output Distributions
- Selective Encoding for Abstractive Sentence Summarization
- Learning to Remember Rare Events
Cited by in corpus (8)
- Big Bird: Transformers for Longer Sequences
- An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics
- Efficient Adaptation of Pretrained Transformers for Abstractive Summarization
- HTLM: Hyper-Text Pre-Training and Prompting of Language Models
- In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
- SMILE: Evaluation and Domain Adaptation for Social Media Language Understanding
- Improving unsupervised anomaly localization by applying multi-scale memories to autoencoders
- CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering