Ensemble-Based Deep Reinforcement Learning for Chatbots

Abstract

Trainable chatbots that exhibit fluent and human-like conversations remain a big challenge in artificial intelligence. Deep Reinforcement Learning (DRL) is promising for addressing this challenge, but its successful application remains an open question. This article describes a novel ensemble-based approach applied to value-based DRL chatbots, which use finite action sets as a form of meaning representation. In our approach, while dialogue actions are derived from sentence clustering, the training datasets in our ensemble are derived from dialogue clustering. The latter aim to induce specialised agents that learn to interact in a particular style. In order to facilitate neural chatbot training using our proposed approach, we assume dialogue data in raw text only -- without any manually-labelled data. Experimental results using chitchat data reveal that (1) near human-like dialogue policies can be induced, (2) generalisation to unseen data is a difficult problem, and (3) training an ensemble of chatbot agents is essential for improved performance over using a single agent. In addition to evaluations using held-out data, our results are further supported by a human evaluation that rated dialogues in terms of fluency, engagingness and consistency -- which revealed that our proposed dialogue rewards strongly correlate with human judgements.

arXiv admin note: text overlap with arXiv:1908.10331

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distributed Representations of Words and Phrases and their Compositionality
• Sequence to Sequence Learning with Neural Networks
• Distributed Representations of Sentences and Documents
• Dueling Network Architectures for Deep Reinforcement Learning
• A Neural Conversational Model
• Universal Sentence Encoder
• Deep Reinforcement Learning: An Overview
• Personalizing Dialogue Agents: I have a dog, do you have pets too?
• Why We Need New Evaluation Metrics for NLG
• Ensemble-Based Deep Reinforcement Learning for Chatbots
• Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation
• A Data-Efficient Deep Learning Approach for Deployable Multimodal Social Robots
• A Deep Reinforcement Learning Chatbot (Short Version)
• An Ensemble Model with Ranking for Social Dialogue

Cited by in corpus (5)

• Ensemble deep learning: A review
• A Literature Survey of Recent Advances in Chatbots
• Ensemble-Based Deep Reinforcement Learning for Chatbots
• Digital Twin: Values, Challenges and Enablers
• AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Tracking