Continuously Learning Neural Dialogue Management

Abstract

We describe a two-step approach for dialogue management in task-oriented spoken dialogue systems. A unified neural network framework is proposed to enable the system to first learn by supervision from a set of dialogue data and then continuously improve its behaviour via reinforcement learning, all using gradient-based algorithms on one single model. The experiments demonstrate the supervised model's effectiveness in the corpus-based evaluation, with user simulation, and with paid human subjects. The use of reinforcement learning further improves the model's performance in both interactive settings, especially under higher-noise conditions.

References in corpus (3)

• A Neural Conversational Model
• Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
• Strategic Dialogue Management via Deep Reinforcement Learning

Cited by in corpus (32)

• Deep Reinforcement Learning for Dialogue Generation
• Adversarial Learning for Neural Dialogue Generation
• A Deep Reinforcement Learning Chatbot
• A Survey of Available Corpora for Building Data-Driven Dialogue Systems
• Why We Need New Evaluation Metrics for NLG
• A User Simulator for Task-Completion Dialogues
• Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning
• Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
• End-to-End Task-Completion Neural Dialogue Systems
• Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
• Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
• LSTM vs. GRU vs. Bidirectional RNN for script generation
• End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient
• Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems
• Budgeted Policy Learning for Task-Oriented Dialogue Systems
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management
• Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability
• Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning
• Deep Active Learning for Dialogue Generation
• Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning
• Teaching Machines to Converse
• Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning
• Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner
• An Overview of Natural Language State Representation for Reinforcement Learning
• Automatic Curriculum Learning With Over-repetition Penalty for Dialogue Policy Learning
• Diluted Near-Optimal Expert Demonstrations for Guiding Dialogue Stochastic Policy Optimisation
• Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning
• Towards End-to-End Learning for Efficient Dialogue Agent by Modeling Looking-ahead Ability
• Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness
• Data-Efficient Methods for Dialogue Systems
• Generative Dialog Policy for Task-oriented Dialog Systems