Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey

Abstract

Neural text generation metamorphosed into several critical natural language applications ranging from text completion to free form narrative generation. In order to progress research in text generation, it is critical to absorb the existing research works and position ourselves in this massively growing field. Specifically, this paper surveys the fundamental components of modeling approaches relaying task agnostic impacts across various generation tasks such as storytelling, summarization, translation etc., In this context, we present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them. Thereby, we deliver a one-stop destination for researchers in the field to facilitate a perspective on where to situate their work and how it impacts other closely related generation tasks.

arXiv comment16 pages

References in corpus (25)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• BERTScore: Evaluating Text Generation with BERT
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• A Deep Reinforced Model for Abstractive Summarization
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Improved Image Captioning via Policy Gradient optimization of SPIDEr
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Evaluation of Text Generation: A Survey
• A Survey of Domain Adaptation for Neural Machine Translation
• DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
• Theory and Experiments on Vector Quantized Autoencoders
• Neural Text Generation: Past, Present and Beyond
• Semi-Autoregressive Training Improves Mask-Predict Decoding
• Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
• Explaining Question Answering Models through Text Generation
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
• GO FIGURE: A Meta Evaluation of Factuality in Summarization
• Neural Machine Translation with Gumbel-Greedy Decoding
• NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
• Cascaded Text Generation with Markov Transformers
• Induction and Reference of Entities in a Visual Story
• A Brief Survey of Multilingual Neural Machine Translation