Training Neural Response Selection for Task-Oriented Dialogue Systems

Abstract

Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method.

ACL 2019 long paper

References in corpus (14)

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• A Neural Conversational Model
• Universal Sentence Encoder
• Self-Normalizing Neural Networks
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Revisiting Small Batch Training for Deep Neural Networks
• Passage Re-ranking with BERT
• Efficient Natural Language Response Suggestion for Smart Reply
• An Information Retrieval Approach to Short Text Conversation
• Smart Reply: Automated Response Suggestion for Email
• Simple Applications of BERT for Ad Hoc Document Retrieval
• Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
• Scale out for large minibatch SGD: Residual network training on ImageNet-1K with improved accuracy and reduced time to train
• Sentence Pair Scoring: Towards Unified Framework for Text Comprehension

Cited by in corpus (17)

• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval
• A Repository of Conversational Datasets
• Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech
• Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems
• Pre-Training for Query Rewriting in A Spoken Language Understanding System
• Pretraining the Noisy Channel Model for Task-Oriented Dialogue
• ConveRT for FAQ Answering
• Is this Dialogue Coherent? Learning from Dialogue Acts and Entities
• Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging
• ConvFiT: Conversational Fine-Tuning of Pretrained Language Models
• Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
• A Conditional Generative Matching Model for Multi-lingual Reply Suggestion
• Extreme Multi-label Learning for Semantic Matching in Product Search
• Language Scaling for Universal Suggested Replies Model
• Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care
• Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration