A Deep Reinforcement Learning Chatbot

Abstract

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including template-based models, bag-of-words models, sequence-to-sequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data.

40 pages, 9 figures, 11 tables

References in corpus (6)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• A Convolutional Neural Network for Modelling Sentences
• Deep Learning for Answer Sentence Selection
• Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
• Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models

Cited by in corpus (3)

• A Deep Reinforcement Learning Chatbot (Short Version)
• Science Driven Innovations Powering Mobile Product: Cloud AI vs. Device AI Solutions on Smart Device
• The Eigenoption-Critic Framework