Challenges in Data-to-Document Generation

Abstract

Recent neural models have shown significant progress on the problem of generating short descriptive texts conditioned on a small number of database records. In this work, we suggest a slightly more difficult data-to-text generation task, and investigate how effective current approaches are on this task. In particular, we introduce a new, large-scale corpus of data records paired with descriptive documents, propose a series of extractive evaluation methods for analyzing performance, and obtain baseline results using current neural generation methods. Experiments show that these models produce fluent text, but fail to convincingly approximate human-generated documents. Moreover, even templated baselines exceed the performance of these neural models on some metrics, though copy- and reconstruction-based extensions lead to noticeable improvements.

EMNLP 2017

References in corpus (9)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Effective Approaches to Attention-based Neural Machine Translation
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Pointer Sentinel Mixture Models
• Classifying Relations by Ranking with Convolutional Neural Networks
• Adversarial Evaluation of Dialogue Models
• Learning to generate one-sentence biographies from Wikidata
• Reference-Aware Language Models

Cited by in corpus (24)

• Big Bird: Transformers for Longer Sequences
• Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
• Evaluation of Text Generation: A Survey
• Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
• Assessing The Factual Accuracy of Generated Text
• Clinically Accurate Chest X-Ray Report Generation
• OpenNMT: Neural Machine Translation Toolkit
• Describing a Knowledge Base
• Neural Language Generation: Formulation, Methods, and Evaluation
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• Story Realization: Expanding Plot Events into Sentences
• Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation
• Controlling Decoding for More Abstractive Summaries with Copy-Based Networks
• Automatic Article Commenting: the Task and Dataset
• Cascaded Text Generation with Markov Transformers
• Operations Guided Neural Networks for High Fidelity Data-To-Text Generation
• Improved Natural Language Generation via Loss Truncation
• Discourse-Aware Neural Rewards for Coherent Text Generation
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• Long-range and hierarchical language predictions in brains and algorithms
• WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation
• Constructing Explainable Opinion Graphs from Review
• Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation