Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey

Abstract

Neural text generation metamorphosed into several critical natural language applications ranging from text completion to free form narrative generation. In order to progress research in text generation, it is critical to absorb the existing research works and position ourselves in this massively growing field. Specifically, this paper surveys the fundamental components of modeling approaches relaying task agnostic impacts across various generation tasks such as storytelling, summarization, translation etc., In this context, we present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them. Thereby, we deliver a one-stop destination for researchers in the field to facilitate a perspective on where to situate their work and how it impacts other closely related generation tasks.

16 pages

References in corpus (11)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Evaluation of Text Generation: A Survey
• Semi-Autoregressive Training Improves Mask-Predict Decoding
• Explaining Question Answering Models through Text Generation
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
• GO FIGURE: A Meta Evaluation of Factuality in Summarization
• Neural Machine Translation with Gumbel-Greedy Decoding
• Cascaded Text Generation with Markov Transformers
• NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints