End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning

Abstract

This paper presents a model for end-to-end learning of task-oriented dialog systems. The main component of the model is a recurrent neural network (an LSTM), which maps from raw dialog history directly to a distribution over system actions. The LSTM automatically infers a representation of dialog history, which relieves the system developer of much of the manual feature engineering of dialog state. In addition, the developer can provide software that expresses business rules and provides access to programmatic APIs, enabling the LSTM to take actions in the real world on behalf of the user. The LSTM can be optimized using supervised learning (SL), where a domain expert provides example dialogs which the LSTM should imitate; or using reinforcement learning (RL), where the system improves by interacting directly with end users. Experiments show that SL and RL are complementary: SL alone can derive a reasonable initial policy from a small number of training dialogs; and starting RL optimization with a policy trained with SL substantially accelerates the learning rate of RL.

References in corpus (1)

• ADADELTA: An Adaptive Learning Rate Method

Cited by in corpus (35)

• A Survey on Dialogue Systems: Recent Advances and New Frontiers
• Efficient Natural Language Response Suggestion for Smart Reply
• Natural Language Processing Advancements By Deep Learning: A Survey
• A Survey of Available Corpora for Building Data-Driven Dialogue Systems
• A User Simulator for Task-Completion Dialogues
• Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access
• End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient
• Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems
• Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models
• Personalizing a Dialogue System with Transfer Reinforcement Learning
• Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems
• An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking
• Fine Grained Knowledge Transfer for Personalized Task-oriented Dialogue Systems
• Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models
• Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use
• Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability
• Sample-efficient Deep Reinforcement Learning for Dialog Control
• Reference-Aware Language Models
• Deep Active Learning for Dialogue Generation
• SUMBT+LaRL: Effective Multi-domain End-to-end Neural Task-oriented Dialog System
• Cross-domain Dialogue Policy Transfer via Simultaneous Speech-act and Slot Alignment
• An Overview of Natural Language State Representation for Reinforcement Learning
• Adversarial Learning of Task-Oriented Neural Dialog Models
• Structured Hierarchical Dialogue Policy with Graph Neural Networks
• Towards End-to-End Learning for Efficient Dialogue Agent by Modeling Looking-ahead Ability
• DialogAct2Vec: Towards End-to-End Dialogue Agent by Multi-Task Representation Learning
• A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots
• Achieving Fluency and Coherency in Task-oriented Dialog
• MEEP: An Open-Source Platform for Human-Human Dialog Collection and End-to-End Agent Training
• Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems
• Hybrid Supervised Reinforced Model for Dialogue Systems
• NE-Table: A Neural key-value table for Named Entities
• Would you Like to Talk about Sports Now? Towards Contextual Topic Suggestion for Open-Domain Conversational Agents
• Incorporating Relevant Knowledge in Context Modeling and Response Generation
• Modeling Multi-Action Policy for Task-Oriented Dialogues