Conversational Contextual Cues: The Case of Personalization and History for Response Ranking

Abstract

We investigate the task of modeling open-domain, multi-turn, unstructured, multi-participant, conversational dialogue. We specifically study the effect of incorporating different elements of the conversation. Unlike previous efforts, which focused on modeling messages and responses, we extend the modeling to long context and participant's history. Our system does not rely on handwritten rules or engineered features; instead, we train deep neural networks on a large conversational dataset. In particular, we exploit the structure of Reddit comments and posts to extract 2.1 billion messages and 133 million conversations. We evaluate our models on the task of predicting the next response in a conversation, and we find that modeling both context and participants improves prediction accuracy.

arXiv comment10 pages, 6 figures

References in corpus (11)

• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• Improving neural networks by preventing co-adaptation of feature detectors
• A Neural Conversational Model
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• A Survey of Available Corpora for Building Data-Driven Dialogue Systems
• Language Understanding for Text-based Games Using Deep Reinforcement Learning
• Attention with Intention for a Neural Network Conversation Model
• deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
• Improved Deep Learning Baselines for Ubuntu Corpus Dialogs
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

Cited by in corpus (19)

• Federated Learning: Strategies for Improving Communication Efficiency
• DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
• Efficient Natural Language Response Suggestion for Smart Reply
• Federated Learning Of Out-Of-Vocabulary Words
• One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles
• Training Neural Response Selection for Task-Oriented Dialogue Systems
• The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service
• Learning Semantic Textual Similarity from Conversations
• Training Data Leakage Analysis in Language Models
• Deep Active Learning for Dialogue Generation
• A Repository of Conversational Datasets
• Teaching Machines to Converse
• Multi-Mention Learning for Reading Comprehension with Neural Cascades
• DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing
• Guiding Variational Response Generator to Exploit Persona
• ConvFiT: Conversational Fine-Tuning of Pretrained Language Models
• DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings
• PolyResponse: A Rank-based Approach to Task-Oriented Dialogue with Application in Restaurant Search and Booking
• Data-Efficient Methods for Dialogue Systems