A Survey of Knowledge-Enhanced Text Generation

Abstract

The goal of text generation is to make machines express in human language. It is one of the most important yet challenging tasks in natural language processing (NLP). Since 2014, various neural encoder-decoder models pioneered by Seq2Seq have been proposed to achieve the goal by learning to map input text to output text. However, the input text alone often provides limited knowledge to generate the desired output, so the performance of text generation is still far from satisfaction in many real-world scenarios. To address this issue, researchers have considered incorporating various forms of knowledge beyond the input text into the generation models. This research direction is known as knowledge-enhanced text generation. In this survey, we present a comprehensive review of the research on knowledge enhanced text generation over the past five years. The main content includes two parts: (i) general methods and architectures for integrating knowledge into text generation; (ii) specific techniques and applications according to different forms of knowledge data. This survey can have broad audiences, researchers and practitioners, in academia and industry.

Accepted by ACM Computing Survey (CSUR) in Jan 2022

References in corpus (58)

• Auto-Encoding Variational Bayes
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Neural Message Passing for Quantum Chemistry
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Convolutional Sequence to Sequence Learning
• Bidirectional Attention Flow for Machine Comprehension
• Pre-trained Models for Natural Language Processing: A Survey
• Unified Language Model Pre-training for Natural Language Understanding and Generation
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• Effective Approaches to Attention-based Neural Machine Translation
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Graph Transformer Networks
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• Topic Aware Neural Response Generation
• Discovering Discrete Latent Topics with Neural Variational Inference
• Harnessing Deep Neural Networks with Logic Rules
• Toward Controlled Generation of Text
• The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
• Plug and Play Language Models: A Simple Approach to Controlled Text Generation
• Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue
• Abstract Meaning Representation for Multi-Document Summarization
• A Survey of Deep Learning Techniques for Neural Machine Translation
• Flexible End-to-End Dialogue System for Knowledge Grounded Conversation
• Pointing the Unknown Words
• Syntax-Directed Attention for Neural Machine Translation
• The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
• Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation
• PaperRobot: Incremental Draft Generation of Scientific Ideas
• Deep Generative Models with Learnable Knowledge Constraints
• Enhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations
• KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
• A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization
• Towards a Continuous Knowledge Learning Engine for Chatbots
• Neural Language Generation: Formulation, Methods, and Evaluation
• Domain Aware Neural Dialog System
• Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
• Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction
• Dict-BERT: Enhancing Language Model Pre-training with Dictionary
• A Neural Topical Expansion Framework for Unstructured Persona-oriented Dialogue Generation
• Fact-based Text Editing
• Recurrent Hierarchical Topic-Guided RNN for Language Generation
• Pre-training Text-to-Text Transformers for Concept-centric Common Sense
• CDL: Curriculum Dual Learning for Emotion-Controllable Response Generation
• Graph-based Multi-hop Reasoning for Long Text Generation
• Entity Commonsense Representation for Neural Abstractive Summarization
• Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
• Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
• Data-to-Text Generation with Style Imitation
• Controllable Paraphrase Generation with a Syntactic Exemplar
• Polite Dialogue Generation Without Parallel Data
• Product-Aware Answer Generation in E-Commerce Question-Answering
• Task-Oriented Conversation Generation Using Heterogeneous Memory Networks
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation
• Towards information-rich, logical text generation with knowledge-enhanced neural models

Cited by in corpus (14)

• Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
• Enhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations
• RepsNet: Combining Vision with Language for Automated Medical Reports
• Knowledge-Aware Semantic Communication System Design and Data Allocation
• SoccerRAG: Multimodal Soccer Information Retrieval via Natural Queries
• Knowledge-Aware Semantic Communication System Design
• LiResolver: License Incompatibility Resolution for Open Source Software
• Fusing Context Into Knowledge Graph for Commonsense Question Answering
• generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation
• TextBox: A Unified, Modularized, and Extensible Framework for Text Generation
• Enhancing Cross-Modal Contextual Congruence for Crowdfunding Success using Knowledge-infused Learning
• Retrieval Enhanced Model for Commonsense Generation
• Sentence-Permuted Paragraph Generation
• Adversarial Machine Learning in Text Analysis and Generation