Dialogue Learning With Human-In-The-Loop
arXiv:1611.09823
Abstract
An important aspect of developing conversational agents is to give a bot the ability to improve through communicating with humans and to learn from the mistakes that it makes. Most research has focused on learning from fixed training sets of labeled data rather than interacting with a dialogue partner in an online fashion. In this paper we explore this direction in a reinforcement learning setting where the bot improves its question-answering ability from feedback a teacher gives following its generated responses. We build a simulator that tests various aspects of such learning in a synthetic environment, and introduce models that work in this regime. Finally, real experiments with Mechanical Turk validate the approach.
Cited by in corpus (20)
- A Survey of Human-in-the-loop for Machine Learning
- Deep Reinforcement Learning: An Overview
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
- Entity-Relation Extraction as Multi-Turn Question Answering
- Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
- Teaching Machines to Describe Images via Natural Language Feedback
- Latent Intention Dialogue Models
- Lifelong and Interactive Learning of Factual Knowledge in Dialogues
- Putting Humans in the Natural Language Processing Loop: A Survey
- Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use
- Chat More If You Like: Dynamic Cue Words Planning to Flow Longer Conversations
- End-to-End Spoken Language Translation
- Teaching Machines to Converse
- Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI
- Countering Language Drift with Seeded Iterated Learning
- Interactive Classification by Asking Informative Questions
- Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions
- Supervised Seeded Iterated Learning for Interactive Language Learning
- Findings from Experiments of On-line Joint Reinforcement Learning of Semantic Parser and Dialogue Manager with real Users