Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI
arXiv:2008.12095
Abstract
Intelligent assistants that follow commands or answer simple questions, such as Siri and Google search, are among the most economically important applications of AI. Future conversational AI assistants promise even greater capabilities and a better user experience through a deeper understanding of the domain, the user, or the user's purposes. But what domain and what methods are best suited to researching and realizing this promise? In this article we argue for the domain of voice document editing and for the methods of model-based reinforcement learning. The primary advantages of voice document editing are that the domain is tightly scoped and that it provides something for the conversation to be about (the document) that is delimited and fully accessible to the intelligent assistant. The advantages of reinforcement learning in general are that its methods are designed to learn from interaction without explicit instruction and that it formalizes the purposes of the assistant. Model-based reinforcement learning is needed in order to genuinely understand the domain of discourse and thereby work efficiently with the user to achieve their goals. Together, voice document editing and model-based reinforcement learning comprise a promising research direction for achieving conversational AI.
Currently under review
References in corpus (44)
- Sequence to Sequence Learning with Neural Networks
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Playing Atari with Deep Reinforcement Learning
- Dueling Network Architectures for Deep Reinforcement Learning
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Gaussian Processes for Data-Efficient Learning in Robotics and Control
- Deep Reinforcement Learning for Dialogue Generation
- Model-Based Reinforcement Learning for Atari
- Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System
- ReasoNet: Learning to Stop Reading in Machine Comprehension
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Building a Conversational Agent Overnight with Dialogue Self-Play
- Individual and Domain Adaptation in Sentence Planning for Dialogue
- Why We Need New Evaluation Metrics for NLG
- The Predictron: End-To-End Learning and Planning
- A User Simulator for Task-Completion Dialogues
- Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
- Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping
- BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
- Personalized Dialogue Generation with Diversified Traits
- ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
- Learning through Dialogue Interactions by Asking Questions
- Learning and Querying Fast Generative Models for Reinforcement Learning
- End-to-End Task-Completion Neural Dialogue Systems
- End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning
- Hierarchical Text Generation and Planning for Strategic Dialogue
- Dialogue Learning With Human-In-The-Loop
- Beating Atari with Natural Language Guided Reinforcement Learning
- Deep Reinforcement Learning with a Natural Language Action Space
- End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient
- Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
- Attention over Parameters for Dialogue Systems
- Multi-Domain Adversarial Learning for Slot Filling in Spoken Language Understanding
- The Effect of Planning Shape on Dyna-style Planning in High-dimensional State Spaces
- Face valuing: Training user interfaces with facial expressions and reinforcement learning
- Learning Robust Dialog Policies in Noisy Environments
- Plato Dialogue System: A Flexible Conversational AI Research Platform
- Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
- Simultaneous Control and Human Feedback in the Training of a Robotic Agent with Actor-Critic Reinforcement Learning
- Towards Multi-Agent Communication-Based Language Learning
- Communicative Capital for Prosthetic Agents
- Model-based Bayesian Reinforcement Learning for Dialogue Management
- Sample-Efficient Model-based Actor-Critic for an Interactive Dialogue Task
- HSCJN: A Holistic Semantic Constraint Joint Network for Diverse Response Generation