A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions

Abstract

One of the hardest problems in the area of Natural Language Processing and Artificial Intelligence is automatically generating language that is coherent and understandable to humans. Teaching machines how to converse as humans do falls under the broad umbrella of Natural Language Generation. Recent years have seen unprecedented growth in the number of research articles published on this subject in conferences and journals both by academic and industry researchers. There have also been several workshops organized alongside top-tier NLP conferences dedicated specifically to this problem. All this activity makes it hard to clearly define the state of the field and reason about its future directions. In this work, we provide an overview of this important and thriving area, covering traditional approaches, statistical approaches and also approaches that use deep neural networks. We provide a comprehensive review towards building open domain dialogue systems, an important application of natural language generation. We find that, predominantly, the approaches for building dialogue systems use seq2seq or language models architecture. Notably, we identify three important areas of further research towards building more effective dialogue systems: 1) incorporating larger context, including conversation context and world knowledge; 2) adding personae or personality in the NLG system; and 3) overcoming dull and generic responses that affect the quality of system-produced responses. We provide pointers on how to tackle these open problems through the use of cognitive architectures that mimic human language understanding and generation capabilities.

References in corpus (34)

• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• A Critical Review of Recurrent Neural Networks for Sequence Learning
• A Neural Conversational Model
• Building Natural-Language Generation Systems
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• Dynamic Coattention Networks For Question Answering
• Deep Reinforcement Learning for Dialogue Generation
• Topic Aware Neural Response Generation
• Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Adversarial Learning for Neural Dialogue Generation
• Personalizing Dialogue Agents: I have a dog, do you have pets too?
• A Persona-Based Neural Conversation Model
• Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
• Hierarchical Recurrent Attention Network for Response Generation
• Attention with Intention for a Neural Network Conversation Model
• Personalized Dialogue Generation with Diversified Traits
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation
• LSTM based Conversation Models
• A Context-aware Natural Language Generator for Dialogue Systems
• Coherent Dialogue with Attention-based Language Models
• Latent Variable Dialogue Models and their Diversity
• Generative Concatenative Nets Jointly Learn to Write and Classify Reviews
• A Conditional Variational Framework for Dialog Generation
• Training Millions of Personalized Dialogue Agents
• Affect-LM: A Neural Language Model for Customizable Affective Text Generation
• Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation
• Deep Active Learning for Dialogue Generation
• soc2seq: Social Embedding meets Conversation Model

Cited by in corpus (10)

• Medical Deep Learning -- A systematic Meta-Review
• Natural Language Interfaces to Data
• A Survey of Document Grounded Dialogue Systems (DGDS)
• Introducing Aspects of Creativity in Automatic Poetry Generation
• The Panacea Threat Intelligence and Active Defense Platform
• Task-oriented Dialogue Systems: performance vs. quality-optima, a review
• Collocation2Text: Controllable Text Generation from Guide Phrases in Russian
• Detecting Asks in SE attacks: Impact of Linguistic and Structural Knowledge
• Deep Learning -- A first Meta-Survey of selected Reviews across Scientific Disciplines, their Commonalities, Challenges and Research Impact
• FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning