Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI

Abstract

Intelligent assistants that follow commands or answer simple questions, such as Siri and Google search, are among the most economically important applications of AI. Future conversational AI assistants promise even greater capabilities and a better user experience through a deeper understanding of the domain, the user, or the user's purposes. But what domain and what methods are best suited to researching and realizing this promise? In this article we argue for the domain of voice document editing and for the methods of model-based reinforcement learning. The primary advantages of voice document editing are that the domain is tightly scoped and that it provides something for the conversation to be about (the document) that is delimited and fully accessible to the intelligent assistant. The advantages of reinforcement learning in general are that its methods are designed to learn from interaction without explicit instruction and that it formalizes the purposes of the assistant. Model-based reinforcement learning is needed in order to genuinely understand the domain of discourse and thereby work efficiently with the user to achieve their goals. Together, voice document editing and model-based reinforcement learning comprise a promising research direction for achieving conversational AI.

Currently under review

References in corpus (17)

• Sequence to Sequence Learning with Neural Networks
• Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System
• Building a Conversational Agent Overnight with Dialogue Self-Play
• Why We Need New Evaluation Metrics for NLG
• Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
• Learning through Dialogue Interactions by Asking Questions
• End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning
• Dialogue Learning With Human-In-The-Loop
• End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient
• Attention over Parameters for Dialogue Systems
• Multi-Domain Adversarial Learning for Slot Filling in Spoken Language Understanding
• Learning Robust Dialog Policies in Noisy Environments
• Plato Dialogue System: A Flexible Conversational AI Research Platform
• Simultaneous Control and Human Feedback in the Training of a Robotic Agent with Actor-Critic Reinforcement Learning
• Communicative Capital for Prosthetic Agents
• Model-based Bayesian Reinforcement Learning for Dialogue Management
• HSCJN: A Holistic Semantic Constraint Joint Network for Diverse Response Generation