Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access

Abstract

This paper proposes KB-InfoBot -- a multi-turn dialogue agent which helps users search Knowledge Bases (KBs) without composing complicated queries. Such goal-oriented dialogue agents typically need to interact with an external database to access real-world knowledge. Previous systems achieved this by issuing a symbolic query to the KB to retrieve entries based on their attributes. However, such symbolic operations break the differentiability of the system and prevent end-to-end training of neural dialogue agents. In this paper, we address this limitation by replacing symbolic queries with an induced "soft" posterior distribution over the KB that indicates which entities the user is interested in. Integrating the soft retrieval process with a reinforcement learner leads to higher task success rate and reward in both simulations and against real users. We also present a fully neural end-to-end agent, trained entirely from user feedback, and discuss its application towards personalized dialogue agents. The source code is available at https://github.com/MiuLab/KB-InfoBot.

Accepted at ACL 2017

References in corpus (6)

• A Network-based End-to-End Trainable Task-oriented Dialogue System
• A User Simulator for Task-Completion Dialogues
• Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning
• End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning
• TensorLog: A Differentiable Deductive Database
• Learning End-to-End Goal-Oriented Dialog

Cited by in corpus (9)

• VIVoNet: Visually-represented, Intent-based, Voice-assisted Networking
• Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability
• Guided Dialog Policy Learning without Adversarial Learning in the Loop
• Interactive Interior Design Recommendation via Coarse-to-fine Multimodal Reinforcement Learning
• Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning
• Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems
• Corpus-Level End-to-End Exploration for Interactive Systems
• End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager
• FinBrain: When Finance Meets AI 2.0