Improving Search through A3C Reinforcement Learning based Conversational Agent
arXiv:1709.05638
Abstract
We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent. Our approach caters to subjective search where the user is seeking digital assets such as images which is fundamentally different from the tasks which have objective and limited search modalities. Labeled conversational data is generally not available in such search tasks and training the agent through human interactions can be time consuming. We propose a stochastic virtual user which impersonates a real user and can be used to sample user behavior efficiently to train the agent which accelerates the bootstrapping of the agent. We develop A3C algorithm based context preserving architecture which enables the agent to provide contextual assistance to the user. We compare the A3C agent with Q-learning and evaluate its performance on average rewards and state values it obtains with the virtual user in validation episodes. Our experiments show that the agent learns to achieve higher rewards and better states.
arXiv comment17 pages, 7 figures
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- A Neural Conversational Model
- Deep Reinforcement Learning for Dialogue Generation
- A Persona-Based Neural Conversation Model
- Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning
- Learning End-to-End Goal-Oriented Dialog
- Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
- Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
- End-to-end optimization of goal-driven and visually grounded dialogue systems
- Task-Oriented Query Reformulation with Reinforcement Learning
- Towards Information-Seeking Agents