A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions

Abstract

One of the hardest problems in the area of Natural Language Processing and Artificial Intelligence is automatically generating language that is coherent and understandable to humans. Teaching machines how to converse as humans do falls under the broad umbrella of Natural Language Generation. Recent years have seen unprecedented growth in the number of research articles published on this subject in conferences and journals both by academic and industry researchers. There have also been several workshops organized alongside top-tier NLP conferences dedicated specifically to this problem. All this activity makes it hard to clearly define the state of the field and reason about its future directions. In this work, we provide an overview of this important and thriving area, covering traditional approaches, statistical approaches and also approaches that use deep neural networks. We provide a comprehensive review towards building open domain dialogue systems, an important application of natural language generation. We find that, predominantly, the approaches for building dialogue systems use seq2seq or language models architecture. Notably, we identify three important areas of further research towards building more effective dialogue systems: 1) incorporating larger context, including conversation context and world knowledge; 2) adding personae or personality in the NLG system; and 3) overcoming dull and generic responses that affect the quality of system-produced responses. We provide pointers on how to tackle these open problems through the use of cognitive architectures that mimic human language understanding and generation capabilities.

References in corpus (10)

• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Hierarchical Recurrent Attention Network for Response Generation
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Coherent Dialogue with Attention-based Language Models
• Latent Variable Dialogue Models and their Diversity

Cited by in corpus (5)

• A Survey of Document Grounded Dialogue Systems (DGDS)
• Introducing Aspects of Creativity in Automatic Poetry Generation
• The Panacea Threat Intelligence and Active Defense Platform
• Detecting Asks in SE attacks: Impact of Linguistic and Structural Knowledge
• FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning