Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs
arXiv:1910.08435
Abstract
Query-based open-domain NLP tasks require information synthesis from long and diverse web results. Current approaches extractively select portions of web text as input to Sequence-to-Sequence models using methods such as TF-IDF ranking. We propose constructing a local graph structured knowledge base for each query, which compresses the web search information and reduces redundancy. We show that by linearizing the graph into a structured input sequence, models can encode the graph representations within a standard Sequence-to-Sequence setting. For two generative tasks with very long text input, long-form question answering and multi-document summarization, feeding graph representations as input can achieve better performance than using retrieved text portions.
References in corpus (8)
- Semi-Supervised Classification with Graph Convolutional Networks
- fairseq: A Fast, Extensible Toolkit for Sequence Modeling
- Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models
- Learning New Facts From Knowledge Bases With Neural Tensor Networks and Semantic Word Vectors
- Graph-based Neural Multi-Document Summarization
- Abstractive Multi-Document Summarization via Phrase Selection and Merging
- ELI5: Long Form Question Answering
- Lattice CNNs for Matching Based Chinese Question Answering